[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:05 dist-test-slave-gjrz krb5kdc[31756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:05 dist-test-slave-gjrz krb5kdc[31756](info): set up 2 sockets
Aug 12 11:16:05 dist-test-slave-gjrz krb5kdc[31756](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 11:16:06.837841 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:06 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:06.952128 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41263
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:07.061024 31788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:07.061255 31788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:07.061308 31788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:07.065223 31788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:07.065274 31788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:07.065299 31788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:07.065318 31788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:07.065339 31788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:07.070423 31788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:07.071358 31788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:07.072295 31788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:07.082051 31797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.082334 31806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.082720 31796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.083178 31788 server_base.cc:1034] running on GCE node
I20260812 11:16:07.083690 31788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:07.084679 31788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:07.085829 31788 hybrid_clock.cc:648] HybridClock initialized: now 1786533367085779 us; error 68 us; skew 500 ppm
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:07.088280 31788 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:07.089246 31788 webserver.cc:460] Webserver started at http://127.30.255.190:34693/ using document root <none> and password file <none>
I20260812 11:16:07.089751 31788 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:07.089814 31788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:07.089993 31788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:07.091988 31788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "3b0f4a4d560749caa5048111dbb8909a"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "a84a32f85783ba0189b5522c89e0ec92"
server_key_iv: "041dbf5fbf41778ff7570b8cf2a6fdb1"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.092588 31788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "3b0f4a4d560749caa5048111dbb8909a"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "a84a32f85783ba0189b5522c89e0ec92"
server_key_iv: "041dbf5fbf41778ff7570b8cf2a6fdb1"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.096014 31788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:16:07.098201 31817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.099345 31788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:07.099521 31788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "3b0f4a4d560749caa5048111dbb8909a"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "a84a32f85783ba0189b5522c89e0ec92"
server_key_iv: "041dbf5fbf41778ff7570b8cf2a6fdb1"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.099628 31788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:07.115795 31788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:07.118238 31788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:07.118391 31788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:07.126645 31788 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41263
I20260812 11:16:07.126650 31916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41263 every 8 connection(s)
I20260812 11:16:07.127545 31788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:07.130275 31918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:07.135792 31918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a: Bootstrap starting.
I20260812 11:16:07.137356 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31788
I20260812 11:16:07.137472 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:07.138244 31918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:07.138705 31742 external_mini_cluster.cc:1428] Setting key 826018d27da9902ba39f7806a3cac6b8
I20260812 11:16:07.138962 31918 log.cc:824] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:07.140930 31918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a: No bootstrap required, opened a new log
I20260812 11:16:07.145819 31918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } }
I20260812 11:16:07.146032 31918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:07.146134 31918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0f4a4d560749caa5048111dbb8909a, State: Initialized, Role: FOLLOWER
I20260812 11:16:07.146613 31918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } }
I20260812 11:16:07.146750 31918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:07.146819 31918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:07.146919 31918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:07.148106 31918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } }
I20260812 11:16:07.148470 31918 leader_election.cc:302] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b0f4a4d560749caa5048111dbb8909a; no voters:
I20260812 11:16:07.148814 31918 leader_election.cc:288] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:07.148883 31924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:07.149112 31924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [term 1 LEADER]: Becoming Leader. State: Replica: 3b0f4a4d560749caa5048111dbb8909a, State: Running, Role: LEADER
I20260812 11:16:07.149446 31924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } }
I20260812 11:16:07.150056 31918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:07.151396 31927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b0f4a4d560749caa5048111dbb8909a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } } }
I20260812 11:16:07.151551 31927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:07.151803 31929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b0f4a4d560749caa5048111dbb8909a. Latest consensus state: current_term: 1 leader_uuid: "3b0f4a4d560749caa5048111dbb8909a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f4a4d560749caa5048111dbb8909a" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41263 } } }
I20260812 11:16:07.151949 31929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:07.152851 31942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:07.155916 31942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:07.161737 31942 catalog_manager.cc:1347] Generated new cluster ID: ad46bdc6e87f46a8b6ed3378e08f7553
I20260812 11:16:07.161798 31942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:07.165859 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.141873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.30.255.190:41263)
0812 11:16:07.142227 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.142326 (+ 99us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.143573 (+ 1247us) server_negotiation.cc:407] Connection header received
0812 11:16:07.146360 (+ 2787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.146373 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.146695 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.146997 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.149987 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.152111 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.154009 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.154780 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.154936 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.159785 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.159791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.159806 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.159835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.162367 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.162987 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.163000 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.163006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.163136 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.163523 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.163526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.163528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.164004 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.164185 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.164794 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.165061 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260812 11:16:07.179813 31942 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:07.180606 31942 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:07.190009 31942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b0f4a4d560749caa5048111dbb8909a: Generated new TSK 0
I20260812 11:16:07.190738 31942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:07.223989 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:07.335542 31962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:07.335747 31962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:07.335790 31962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:07.339699 31962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:07.339747 31962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:07.339810 31962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:07.344820 31962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:07.345721 31962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:07.346649 31962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:07.355822 31976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.355882 31972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.355952 31962 server_base.cc:1034] running on GCE node
W20260812 11:16:07.355911 31973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.356407 31962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:07.356956 31962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:07.358093 31962 hybrid_clock.cc:648] HybridClock initialized: now 1786533367358071 us; error 34 us; skew 500 ppm
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:07.360626 31962 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:07.361479 31962 webserver.cc:460] Webserver started at http://127.30.255.129:35713/ using document root <none> and password file <none>
I20260812 11:16:07.361982 31962 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:07.362041 31962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:07.362228 31962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:07.364089 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "4abc8ba67e0845f290e7a0632786b6b4"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "68d89c56310709a84cd601e4da89f44a"
server_key_iv: "99c02377d52f6536222b60e85ed0edfb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.364677 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "4abc8ba67e0845f290e7a0632786b6b4"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "68d89c56310709a84cd601e4da89f44a"
server_key_iv: "99c02377d52f6536222b60e85ed0edfb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.368091 31962 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:07.370301 31998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.371438 31962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:07.371541 31962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "4abc8ba67e0845f290e7a0632786b6b4"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "68d89c56310709a84cd601e4da89f44a"
server_key_iv: "99c02377d52f6536222b60e85ed0edfb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.371642 31962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:07.390420 31962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:07.392874 31962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:07.393028 31962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:07.393579 31962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:07.394321 31962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:07.394364 31962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.394403 31962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:07.394431 31962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.405830 31962 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:40907
I20260812 11:16:07.405864 32208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:40907 every 8 connection(s)
I20260812 11:16:07.406613 31962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:07.409250 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31962
I20260812 11:16:07.409350 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:07.409593 31742 external_mini_cluster.cc:1428] Setting key 42f2b67c1b2d238266fc2bcef0a3de60
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:07.420212 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.408212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:33453 (local address 127.30.255.190:41263)
0812 11:16:07.408340 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.408345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.409180 (+ 835us) server_negotiation.cc:407] Connection header received
0812 11:16:07.410537 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.410541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.410599 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.410673 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.412178 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.412690 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.413966 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.414301 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.414403 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.416852 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.416855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.416857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.416880 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.418159 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.418707 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.418710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.418711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.418777 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.419187 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.419189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.419190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.419377 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.419450 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.419985 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.420093 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:07.420928 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.408595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41263 (local address 127.30.255.129:33453)
0812 11:16:07.409002 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:07.409083 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:07.410335 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:07.410847 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:07.410854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:07.411348 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:07.412030 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.412050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.412814 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.412818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.413844 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.413854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.414422 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.414425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.414566 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.415046 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:07.415061 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:07.416688 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:07.418271 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.418278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.418291 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.418622 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.418859 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.418862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.418864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.419099 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.419473 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:07.419480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:07.419780 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 11:16:07.419938 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 11:16:07.420163 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260812 11:16:07.422053 32209 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:07.422277 32209 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:07.422758 32209 heartbeater.cc:511] Master 127.30.255.190:41263 requested a full tablet report, sending...
I20260812 11:16:07.424253 31845 ts_manager.cc:194] Registered new tserver with Master: 4abc8ba67e0845f290e7a0632786b6b4 (127.30.255.129:40907)
I20260812 11:16:07.425557 31845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:33453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:07.436252 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:07.549857 32217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:07.550086 32217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:07.550141 32217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:07.554323 32217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:07.554374 32217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:07.554435 32217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:07.559350 32217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:07.560197 32217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:07.561110 32217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:07.570063 32230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.570065 32232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.569983 32234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.570276 32217 server_base.cc:1034] running on GCE node
I20260812 11:16:07.570722 32217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:07.571250 32217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:07.572443 32217 hybrid_clock.cc:648] HybridClock initialized: now 1786533367572427 us; error 26 us; skew 500 ppm
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:07.575119 32217 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:07.576023 32217 webserver.cc:460] Webserver started at http://127.30.255.130:38361/ using document root <none> and password file <none>
I20260812 11:16:07.576525 32217 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:07.576594 32217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:07.576793 32217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:07.578684 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "aa7fd99698414e57bf89317de3d995f0"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "e2e13e3b528b0caf3d9183f92dbb0f6b"
server_key_iv: "930dfe6be22dfa98ad68b0b96c6d0193"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.579206 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "aa7fd99698414e57bf89317de3d995f0"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "e2e13e3b528b0caf3d9183f92dbb0f6b"
server_key_iv: "930dfe6be22dfa98ad68b0b96c6d0193"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.582898 32217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:16:07.585186 32245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.586309 32217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:07.586409 32217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "aa7fd99698414e57bf89317de3d995f0"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "e2e13e3b528b0caf3d9183f92dbb0f6b"
server_key_iv: "930dfe6be22dfa98ad68b0b96c6d0193"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.586498 32217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:07.604660 32217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:07.607475 32217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:07.607632 32217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:07.608186 32217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:07.608922 32217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:07.608966 32217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.609016 32217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:07.609041 32217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.620138 32217 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:36289
I20260812 11:16:07.620162 32453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:36289 every 8 connection(s)
I20260812 11:16:07.620934 32217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:07.621797 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 32217
I20260812 11:16:07.621908 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:07.622149 31742 external_mini_cluster.cc:1428] Setting key c8cb141178a1268517bba9d307912541
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:07.635365 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.623330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:40247 (local address 127.30.255.190:41263)
0812 11:16:07.623444 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.623449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.624358 (+ 909us) server_negotiation.cc:407] Connection header received
0812 11:16:07.625887 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.625890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.625945 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.626016 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.627149 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.627703 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.629326 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.629634 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.629714 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.632320 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.632322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.632325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.632348 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.633548 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.634050 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.634052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.634054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.634124 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.634366 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.634368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.634369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.634523 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.634591 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.635102 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.635195 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:16:07.636255 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.623750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41263 (local address 127.30.255.130:40247)
0812 11:16:07.624200 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:07.624251 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:07.625685 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:07.626171 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:07.626177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:07.626493 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:07.627010 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.627029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.627823 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.627826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.629211 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.629223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.629781 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.629785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.629989 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.630491 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:07.630514 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:07.632159 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:07.633706 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.633712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.633729 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.633971 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.634187 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.634188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.634190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.634302 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.634598 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:07.634602 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:07.634892 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 11:16:07.635043 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 11:16:07.635346 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":99,"threads_started":1}
I20260812 11:16:07.637498 32457 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41263
I20260812 11:16:07.637707 32457 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:07.638146 32457 heartbeater.cc:511] Master 127.30.255.190:41263 requested a full tablet report, sending...
I20260812 11:16:07.639091 31845 ts_manager.cc:194] Registered new tserver with Master: aa7fd99698414e57bf89317de3d995f0 (127.30.255.130:36289)
I20260812 11:16:07.639650 31845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:40247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:07.650664 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:07.762971 32463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:07.763185 32463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:07.763224 32463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:07.767236 32463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:07.767318 32463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:07.767380 32463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:07.772279 32463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:07.773101 32463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:07.773957 32463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:07.782948 32471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:07.782884 32470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.783150 32463 server_base.cc:1034] running on GCE node
W20260812 11:16:07.783303 32473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:07.783660 32463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:07.784212 32463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:07.785387 32463 hybrid_clock.cc:648] HybridClock initialized: now 1786533367785363 us; error 27 us; skew 500 ppm
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:07.787911 32463 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:07.788833 32463 webserver.cc:460] Webserver started at http://127.30.255.131:44663/ using document root <none> and password file <none>
I20260812 11:16:07.789325 32463 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:07.789405 32463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:07.789599 32463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:07.791513 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "61029af662af4b5e81b8e6757d455d75"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "7a5d1e0c1887e6250f55a1b3c802d8ac"
server_key_iv: "e392a016c584661d43b778e7bcaa3fdb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.792068 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "61029af662af4b5e81b8e6757d455d75"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "7a5d1e0c1887e6250f55a1b3c802d8ac"
server_key_iv: "e392a016c584661d43b778e7bcaa3fdb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.795485 32463 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:16:07.797716 32507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.798805 32463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:07.798910 32463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "61029af662af4b5e81b8e6757d455d75"
format_stamp: "Formatted at 2026-08-12 11:16:07 on dist-test-slave-gjrz"
server_key: "7a5d1e0c1887e6250f55a1b3c802d8ac"
server_key_iv: "e392a016c584661d43b778e7bcaa3fdb"
server_key_version: "encryptionkey@0"
I20260812 11:16:07.799000 32463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:07.818197 32463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:07.820691 32463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:07.820847 32463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:07.821347 32463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:07.822089 32463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:07.822135 32463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.822185 32463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:07.822214 32463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:07.835345 32463 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:33849
I20260812 11:16:07.835374 32713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:33849 every 8 connection(s)
I20260812 11:16:07.836107 32463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:07.836941 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 32463
I20260812 11:16:07.837046 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:07.837291 31742 external_mini_cluster.cc:1428] Setting key 5077342632adcc0f257f8b99e228f286
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:07.848666 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.837890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:36103 (local address 127.30.255.190:41263)
0812 11:16:07.838009 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.838013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.838942 (+ 929us) server_negotiation.cc:407] Connection header received
0812 11:16:07.840271 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.840274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.840327 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.840396 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.841502 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.842027 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.843109 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.843370 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.843458 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.845737 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.845740 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.845743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.845770 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.846850 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.847317 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.847320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.847322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.847398 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.847697 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.847701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.847702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.847872 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.847953 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.848397 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.848510 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:16:07.849329 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.838294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41263 (local address 127.30.255.131:36103)
0812 11:16:07.838816 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:07.838868 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:07.840087 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:07.840567 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:07.840574 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:07.840899 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:07.841371 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.841386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.842140 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.842143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.842990 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:07.843000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.843498 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.843502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:07.843706 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.844239 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:07.844255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:07.845578 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:07.846951 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.846957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.846970 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.847201 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.847478 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:07.847482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:07.847484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:07.847613 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:07.847966 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:07.847972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:07.848185 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 11:16:07.848338 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 11:16:07.848628 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":168,"threads_started":1}
I20260812 11:16:07.850363 32715 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41263
I20260812 11:16:07.850601 32715 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:07.851137 32715 heartbeater.cc:511] Master 127.30.255.190:41263 requested a full tablet report, sending...
I20260812 11:16:07.852135 31845 ts_manager.cc:194] Registered new tserver with Master: 61029af662af4b5e81b8e6757d455d75 (127.30.255.131:33849)
I20260812 11:16:07.852676 31845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:36103
I20260812 11:16:07.862696 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:07.888347 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.880096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.30.255.129:40907)
0812 11:16:07.880374 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.880387 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.880408 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:16:07.880472 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.880477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.880641 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.880775 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.881390 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.882139 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.882871 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.883286 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.883396 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.885396 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.885404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.885415 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.885436 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.886845 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.887187 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.887191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.887192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.887289 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.887560 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.887562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.887563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.887769 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.887905 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.888044 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.888170 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":108,"threads_started":1}
W20260812 11:16:07.889179 32043 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38642
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:16:07 dist-test-slave-gjrz krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:07.914287 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:07.906992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.30.255.129:40907)
0812 11:16:07.907130 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:07.907136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:07.907237 (+ 101us) server_negotiation.cc:407] Connection header received
0812 11:16:07.907375 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:07.907378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:07.907423 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:07.907486 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:07.908169 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.908674 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.909395 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:07.909615 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:07.909682 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:07.911743 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:07.911746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:07.911748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:07.911770 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:07.912965 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.913336 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.913339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.913340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.913466 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:07.913677 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:07.913679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:07.913680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:07.913830 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:07.913923 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:07.914075 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 11:16:07.914167 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:16:07.915189 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 31962
I20260812 11:16:07.933552 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 32217
I20260812 11:16:07.956683 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 32463
I20260812 11:16:07.998508 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 31788
2026-08-12T11:16:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2424 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:08 dist-test-slave-gjrz krb5kdc[32742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:08 dist-test-slave-gjrz krb5kdc[32742](info): set up 2 sockets
Aug 12 11:16:08 dist-test-slave-gjrz krb5kdc[32742](info): commencing operation
krb5kdc: starting...
W20260812 11:16:09.204893 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:09.317544 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33469
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:33469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:09.422358 308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:09.422575 308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:09.422638 308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:09.426473 308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:09.426522 308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:09.426548 308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:09.426569 308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:09.426604 308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:09.431484 308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:33469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:09.432319 308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:09.433190 308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:09.440907 321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:09.440937 327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:09.440888 322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:09.441051 308 server_base.cc:1034] running on GCE node
I20260812 11:16:09.441731 308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:09.442683 308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:09.443830 308 hybrid_clock.cc:648] HybridClock initialized: now 1786533369443804 us; error 39 us; skew 500 ppm
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:09.446360 308 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:09.447134 308 webserver.cc:460] Webserver started at http://127.30.255.190:41129/ using document root <none> and password file <none>
I20260812 11:16:09.447641 308 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:09.447700 308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:09.447856 308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:09.449663 308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "242982758f034da89cfc31754a91da2b"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "bfb7dc82b9bb1c90816070a4cf9661d6"
server_key_iv: "e9db5b5125c03ca2dd0474e08b2fc7d5"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.450131 308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "242982758f034da89cfc31754a91da2b"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "bfb7dc82b9bb1c90816070a4cf9661d6"
server_key_iv: "e9db5b5125c03ca2dd0474e08b2fc7d5"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.454524 308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:16:09.456707 336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.457826 308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:09.457924 308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "242982758f034da89cfc31754a91da2b"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "bfb7dc82b9bb1c90816070a4cf9661d6"
server_key_iv: "e9db5b5125c03ca2dd0474e08b2fc7d5"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.458016 308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:09.469159 308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:09.471611 308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:09.471760 308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:09.478281 308 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:33469
I20260812 11:16:09.478299 429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:33469 every 8 connection(s)
I20260812 11:16:09.479103 308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:09.481760 430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:09.482235 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 308
I20260812 11:16:09.482349 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:09.482623 31742 external_mini_cluster.cc:1428] Setting key 959df6a8939136baab4a5a8ee5bc4bfc
I20260812 11:16:09.488335 430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b: Bootstrap starting.
I20260812 11:16:09.490795 430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b: Neither blocks nor log segments found. Creating new log.
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:09.491658 430 log.cc:824] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:09.494030 430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b: No bootstrap required, opened a new log
I20260812 11:16:09.498281 430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } }
I20260812 11:16:09.498512 430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:09.498553 430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242982758f034da89cfc31754a91da2b, State: Initialized, Role: FOLLOWER
I20260812 11:16:09.498971 430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } }
I20260812 11:16:09.499109 430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:09.499156 430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:09.499234 430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:09.500208 430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } }
I20260812 11:16:09.500552 430 leader_election.cc:302] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242982758f034da89cfc31754a91da2b; no voters:
I20260812 11:16:09.500883 430 leader_election.cc:288] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:09.501051 440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:09.501297 440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [term 1 LEADER]: Becoming Leader. State: Replica: 242982758f034da89cfc31754a91da2b, State: Running, Role: LEADER
I20260812 11:16:09.501628 440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } }
I20260812 11:16:09.502141 430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:09.502835 443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242982758f034da89cfc31754a91da2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } } }
I20260812 11:16:09.502844 440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242982758f034da89cfc31754a91da2b. Latest consensus state: current_term: 1 leader_uuid: "242982758f034da89cfc31754a91da2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242982758f034da89cfc31754a91da2b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33469 } } }
I20260812 11:16:09.503252 440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:09.502974 443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:09.503753 454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:09.504398 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:09.483893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.30.255.190:33469)
0812 11:16:09.484255 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:09.484270 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:09.484304 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:16:09.484953 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:09.484970 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:09.485267 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:09.485522 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:09.486327 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.487360 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.488465 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.488910 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.489019 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:09.491873 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:09.491879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:09.491913 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:09.491941 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:09.500549 (+ 8608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.501039 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.501045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.501051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.501289 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.501630 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.501634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.501635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.502139 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:09.502322 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:09.503164 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 11:16:09.503491 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":92,"threads_started":1}
I20260812 11:16:09.507450 454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:09.512578 454 catalog_manager.cc:1347] Generated new cluster ID: 8d46cc3978624d308d60bac80093be3b
I20260812 11:16:09.512646 454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:09.530903 454 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:09.531775 454 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:09.543713 454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242982758f034da89cfc31754a91da2b: Generated new TSK 0
I20260812 11:16:09.544283 454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:09.570096 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:09.680157 469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:09.680379 469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:09.680435 469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:09.684337 469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:09.684386 469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:09.684461 469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:09.689435 469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:09.690315 469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:09.691216 469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:09.699790 483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:09.699867 478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:09.699877 477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:09.700217 469 server_base.cc:1034] running on GCE node
I20260812 11:16:09.700541 469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:09.701082 469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:09.702225 469 hybrid_clock.cc:648] HybridClock initialized: now 1786533369702210 us; error 24 us; skew 500 ppm
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:09.704836 469 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:09.705588 469 webserver.cc:460] Webserver started at http://127.30.255.129:45351/ using document root <none> and password file <none>
I20260812 11:16:09.706074 469 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:09.706146 469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:09.706512 469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:09.708462 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "010c49306c9e4bb3b24069b6710e653d"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "57592569ea9a3e37d22ffdbb2a57cdb2"
server_key_iv: "109ac79c37302ca6e5ff5e1921136359"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.708976 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "010c49306c9e4bb3b24069b6710e653d"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "57592569ea9a3e37d22ffdbb2a57cdb2"
server_key_iv: "109ac79c37302ca6e5ff5e1921136359"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.712430 469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:09.714566 501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.715734 469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:09.715835 469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "010c49306c9e4bb3b24069b6710e653d"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "57592569ea9a3e37d22ffdbb2a57cdb2"
server_key_iv: "109ac79c37302ca6e5ff5e1921136359"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.715920 469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:09.729643 469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:09.732453 469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:09.732621 469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:09.733124 469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:09.733856 469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:09.733901 469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.733947 469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:09.733978 469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.741905 469 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:34987
I20260812 11:16:09.741956 725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:34987 every 8 connection(s)
I20260812 11:16:09.742697 469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:09.745220 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 469
I20260812 11:16:09.745328 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:09.745589 31742 external_mini_cluster.cc:1428] Setting key 7d730f43c0b0141df805d791007de798
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:09.756402 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:09.744303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:58845 (local address 127.30.255.190:33469)
0812 11:16:09.744430 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:09.744435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:09.745216 (+ 781us) server_negotiation.cc:407] Connection header received
0812 11:16:09.746581 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:09.746584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:09.746628 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:09.746693 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:09.747878 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.748574 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.750128 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.750430 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.750525 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:09.753014 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:09.753017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:09.753019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:09.753048 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:09.754269 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.754813 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.754815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.754816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.754882 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.755231 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.755233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.755234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.755421 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:09.755493 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:09.756130 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 11:16:09.756250 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:16:09.757205 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:09.744664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33469 (local address 127.30.255.129:58845)
0812 11:16:09.745083 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:09.745122 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:09.746371 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:09.746885 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:09.746894 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:09.747252 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:09.747725 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:09.747738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.748689 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.748693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:09.749991 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:09.750003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.750560 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.750563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:09.750701 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:09.751178 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:09.751193 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:09.752856 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:09.754381 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:09.754387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:09.754401 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:09.754728 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:09.754959 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:09.754962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:09.754964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:09.755147 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:09.755523 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:09.755530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:09.755811 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 11:16:09.756050 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 11:16:09.756360 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260812 11:16:09.758385 727 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:09.758620 727 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:09.759107 727 heartbeater.cc:511] Master 127.30.255.190:33469 requested a full tablet report, sending...
I20260812 11:16:09.760725 367 ts_manager.cc:194] Registered new tserver with Master: 010c49306c9e4bb3b24069b6710e653d (127.30.255.129:34987)
I20260812 11:16:09.762008 367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:58845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:09.772185 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:09.876847 738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:09.877079 738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:09.877130 738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:09.880963 738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:09.881008 738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:09.881079 738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:09.885941 738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:09.886755 738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:09.887665 738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:09.896088 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:09.896082 746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:09.896320 738 server_base.cc:1034] running on GCE node
W20260812 11:16:09.896083 753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:09.896880 738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:09.897398 738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:09.898547 738 hybrid_clock.cc:648] HybridClock initialized: now 1786533369898527 us; error 29 us; skew 500 ppm
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:09.901017 738 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:09.901782 738 webserver.cc:460] Webserver started at http://127.30.255.130:39085/ using document root <none> and password file <none>
I20260812 11:16:09.902273 738 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:09.902326 738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:09.902534 738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:09.904479 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "f99c08067d444b6c92e70141989ab814"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "064a45f0cb4e99b394eda2886183eddf"
server_key_iv: "962e7ce333cf5f66873af755110b573a"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.904971 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "f99c08067d444b6c92e70141989ab814"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "064a45f0cb4e99b394eda2886183eddf"
server_key_iv: "962e7ce333cf5f66873af755110b573a"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.908259 738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 11:16:09.910375 763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.911525 738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:09.911629 738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "f99c08067d444b6c92e70141989ab814"
format_stamp: "Formatted at 2026-08-12 11:16:09 on dist-test-slave-gjrz"
server_key: "064a45f0cb4e99b394eda2886183eddf"
server_key_iv: "962e7ce333cf5f66873af755110b573a"
server_key_version: "encryptionkey@0"
I20260812 11:16:09.911711 738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:09.933173 738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:09.935525 738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:09.935676 738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:09.936170 738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:09.936877 738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:09.936923 738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.936970 738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:09.937000 738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:09.944937 738 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:36143
I20260812 11:16:09.944960 993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:36143 every 8 connection(s)
I20260812 11:16:09.945679 738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:09.947096 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 738
I20260812 11:16:09.947197 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:09.947481 31742 external_mini_cluster.cc:1428] Setting key 2c606fdae164b399bec788a24ba9c7f5
Aug 12 11:16:09 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:09.959026 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:09.947235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:36755 (local address 127.30.255.190:33469)
0812 11:16:09.947388 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:09.947393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:09.948187 (+ 794us) server_negotiation.cc:407] Connection header received
0812 11:16:09.949684 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:09.949686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:09.949728 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:09.949786 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:09.951132 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.951893 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.952945 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.953183 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.953271 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:09.955773 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:09.955776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:09.955778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:09.955800 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:09.957061 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.957602 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.957604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.957605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.957670 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:09.958001 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:09.958002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:09.958003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:09.958161 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:09.958227 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:09.958819 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 11:16:09.958912 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:16:09.959829 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:09.947604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33469 (local address 127.30.255.130:36755)
0812 11:16:09.948048 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:09.948099 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:09.949451 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:09.949966 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:09.949974 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:09.950394 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:09.951003 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:09.951018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.952000 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.952002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:09.952844 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:09.952853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:09.953337 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:09.953342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:09.953486 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:09.953983 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:09.953998 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:09.955617 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:09.957176 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:09.957182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:09.957196 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:09.957518 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:09.957744 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:09.957747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:09.957749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:09.957920 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:09.958255 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:09.958261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:09.958534 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 11:16:09.958777 (+ 243us) client_negotiation.cc:240] Negotiation successful
0812 11:16:09.959070 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260812 11:16:09.960767 994 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33469
I20260812 11:16:09.960988 994 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:09.961443 994 heartbeater.cc:511] Master 127.30.255.190:33469 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:16:09.962409 367 ts_manager.cc:194] Registered new tserver with Master: f99c08067d444b6c92e70141989ab814 (127.30.255.130:36143)
I20260812 11:16:09.962937 367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:36755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:09.976070 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:10.082727 1000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:10.082947 1000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:10.083005 1000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:10.086863 1000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:10.086912 1000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:10.086990 1000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:10.091965 1000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:10.092844 1000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:10.093719 1000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:10.102507 1020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:10.102581 1015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:10.102584 1016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:10.102911 1000 server_base.cc:1034] running on GCE node
I20260812 11:16:10.103327 1000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:10.103893 1000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:10.105099 1000 hybrid_clock.cc:648] HybridClock initialized: now 1786533370105071 us; error 53 us; skew 500 ppm
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:10.107592 1000 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:10.108381 1000 webserver.cc:460] Webserver started at http://127.30.255.131:46553/ using document root <none> and password file <none>
I20260812 11:16:10.108856 1000 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:10.108914 1000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:10.109090 1000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:10.110991 1000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "d0ff1e1735154b4984f644478969a085"
format_stamp: "Formatted at 2026-08-12 11:16:10 on dist-test-slave-gjrz"
server_key: "0613ef6daed9eef41b7baf16f8f029a6"
server_key_iv: "43977419bede8fe00aa5f775353ecc97"
server_key_version: "encryptionkey@0"
I20260812 11:16:10.111521 1000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "d0ff1e1735154b4984f644478969a085"
format_stamp: "Formatted at 2026-08-12 11:16:10 on dist-test-slave-gjrz"
server_key: "0613ef6daed9eef41b7baf16f8f029a6"
server_key_iv: "43977419bede8fe00aa5f775353ecc97"
server_key_version: "encryptionkey@0"
I20260812 11:16:10.114863 1000 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260812 11:16:10.117121 1043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:10.118194 1000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 11:16:10.118285 1000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "d0ff1e1735154b4984f644478969a085"
format_stamp: "Formatted at 2026-08-12 11:16:10 on dist-test-slave-gjrz"
server_key: "0613ef6daed9eef41b7baf16f8f029a6"
server_key_iv: "43977419bede8fe00aa5f775353ecc97"
server_key_version: "encryptionkey@0"
I20260812 11:16:10.118372 1000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:10.132791 1000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:10.135114 1000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:10.135283 1000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:10.135783 1000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:10.136493 1000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:10.136538 1000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:10.136576 1000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:10.136606 1000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:10.145824 1000 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:34767
I20260812 11:16:10.145846 1270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:34767 every 8 connection(s)
I20260812 11:16:10.146576 1000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:10.151335 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1000
I20260812 11:16:10.151438 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:10.151705 31742 external_mini_cluster.cc:1428] Setting key 2c39c54784f3c4de3151853cd2da038c
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:10.159296 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.148179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:36663 (local address 127.30.255.190:33469)
0812 11:16:10.148295 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.148298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.149015 (+ 717us) server_negotiation.cc:407] Connection header received
0812 11:16:10.150259 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.150262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.150323 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.150394 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:10.151517 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.152219 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.153773 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.154008 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.154097 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.156277 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:10.156280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:10.156282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:10.156305 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:10.157394 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.157855 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.157859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.157861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.157949 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.158259 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.158263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.158264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.158435 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:10.158522 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.158990 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.159116 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:16:10.159963 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.148469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33469 (local address 127.30.255.131:36663)
0812 11:16:10.148886 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.148924 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.150062 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.150567 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.150573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.150905 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:10.151375 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.151390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.152378 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.152381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.153647 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.153659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.154150 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.154155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.154325 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.154781 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:10.154795 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:10.156122 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:10.157503 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:10.157509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:10.157522 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:10.157759 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:10.158035 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:10.158038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:10.158040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:10.158169 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:10.158538 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:10.158545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:10.158761 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.158919 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.159204 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":101,"threads_started":1}
I20260812 11:16:10.160917 1271 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33469
I20260812 11:16:10.161120 1271 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:10.161566 1271 heartbeater.cc:511] Master 127.30.255.190:33469 requested a full tablet report, sending...
I20260812 11:16:10.162505 367 ts_manager.cc:194] Registered new tserver with Master: d0ff1e1735154b4984f644478969a085 (127.30.255.131:34767)
I20260812 11:16:10.163019 367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:36663
I20260812 11:16:10.165324 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:16:10.174885 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.168433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41528 (local address 127.30.255.190:33469)
0812 11:16:10.168546 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.168550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.168692 (+ 142us) server_negotiation.cc:407] Connection header received
0812 11:16:10.168785 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.168786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.168820 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.168878 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:10.169449 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.169906 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.170697 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.170914 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.171009 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.172230 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:10.172233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:10.172235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:10.172257 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:10.173478 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.173961 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.173963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.173964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.174029 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.174253 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.174255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.174256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.174438 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:10.174533 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.174676 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.174755 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:10.179153 367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:10.182021 367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:10.195446 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.189905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:34987 (local address 127.0.0.1:47186)
0812 11:16:10.190333 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.190374 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.190503 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.191105 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.191111 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.191137 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:10.191293 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.191302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.192750 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.192753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.193654 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.193664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.194893 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.194897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.195042 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.195086 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.195178 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.195309 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":29,"thread_start_us":119,"threads_started":1}
I20260812 11:16:10.195632 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.190702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.30.255.130:36143)
0812 11:16:10.191136 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.191147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.191170 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:16:10.191244 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.191249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.191467 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.191626 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:10.191857 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.192780 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.194273 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.195023 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.195175 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.195322 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.195389 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.195483 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":134,"threads_started":1}
I20260812 11:16:10.195855 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.190033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:36143 (local address 127.0.0.1:36272)
0812 11:16:10.190623 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.190635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.190717 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.191605 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.191608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.191627 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:10.191721 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.191728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.192941 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.192944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.194145 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.194156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.195160 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.195164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.195246 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.195292 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.195342 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.195398 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":30,"thread_start_us":102,"threads_started":1}
I20260812 11:16:10.195998 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.190048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.30.255.129:34987)
0812 11:16:10.190440 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.190450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.190474 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:16:10.190714 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.190720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.190929 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.191092 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:10.191554 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.192638 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.193773 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.194605 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.194718 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.194835 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.195763 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.195859 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":106,"threads_started":1}
I20260812 11:16:10.196405 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.190647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:34767 (local address 127.0.0.1:42532)
0812 11:16:10.191870 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.191884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.191987 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.192457 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.192460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.192478 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:10.192571 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.192576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.193908 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.193911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.194984 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.194993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.196015 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.196019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.196115 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.196136 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.196212 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.196278 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":1,"thread_start_us":148,"threads_started":1}
I20260812 11:16:10.196448 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.190986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.30.255.131:34767)
0812 11:16:10.191520 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.191531 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.192073 (+ 542us) server_negotiation.cc:407] Connection header received
0812 11:16:10.192145 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.192150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.192316 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.192452 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:10.192679 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.193788 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.195106 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.195895 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.196017 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.196135 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.196240 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.196323 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":123,"threads_started":1}
I20260812 11:16:10.198845 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 335bd56a06f642b89727069cd7405842 (DEFAULT_TABLE table=test-table [id=2a91e91259654b80b54ae4728b862a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:10.198844 592 tablet_service.cc:1467] Processing CreateTablet for tablet 335bd56a06f642b89727069cd7405842 (DEFAULT_TABLE table=test-table [id=2a91e91259654b80b54ae4728b862a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:10.199766 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bd56a06f642b89727069cd7405842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:10.200690 871 tablet_service.cc:1467] Processing CreateTablet for tablet 335bd56a06f642b89727069cd7405842 (DEFAULT_TABLE table=test-table [id=2a91e91259654b80b54ae4728b862a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:10.201583 871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bd56a06f642b89727069cd7405842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:10.201894 592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bd56a06f642b89727069cd7405842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:10.206832 1313 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Bootstrap starting.
I20260812 11:16:10.207887 1314 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: Bootstrap starting.
I20260812 11:16:10.208802 1317 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Bootstrap starting.
I20260812 11:16:10.209525 1313 tablet_bootstrap.cc:654] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:10.210297 1314 tablet_bootstrap.cc:654] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:10.211005 1314 log.cc:824] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:10.211092 1313 log.cc:824] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:10.211561 1317 tablet_bootstrap.cc:654] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:10.212339 1317 log.cc:824] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:10.213045 1314 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: No bootstrap required, opened a new log
I20260812 11:16:10.213233 1314 ts_tablet_manager.cc:1397] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:16:10.213495 1313 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: No bootstrap required, opened a new log
I20260812 11:16:10.213675 1313 ts_tablet_manager.cc:1397] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:16:10.213893 1317 tablet_bootstrap.cc:492] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: No bootstrap required, opened a new log
I20260812 11:16:10.214025 1317 ts_tablet_manager.cc:1397] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 11:16:10.216993 1317 raft_consensus.cc:348] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.217195 1317 raft_consensus.cc:374] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:10.217248 1317 raft_consensus.cc:729] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010c49306c9e4bb3b24069b6710e653d, State: Initialized, Role: FOLLOWER
I20260812 11:16:10.217165 1314 raft_consensus.cc:348] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.217221 1313 raft_consensus.cc:348] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.217433 1313 raft_consensus.cc:374] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:10.217433 1314 raft_consensus.cc:374] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:10.217490 1314 raft_consensus.cc:729] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99c08067d444b6c92e70141989ab814, State: Initialized, Role: FOLLOWER
I20260812 11:16:10.217490 1313 raft_consensus.cc:729] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ff1e1735154b4984f644478969a085, State: Initialized, Role: FOLLOWER
I20260812 11:16:10.217698 1317 consensus_queue.cc:260] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.217902 1313 consensus_queue.cc:260] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.217902 1314 consensus_queue.cc:260] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.218358 727 heartbeater.cc:503] Master 127.30.255.190:33469 was elected leader, sending a full tablet report...
I20260812 11:16:10.218938 1314 ts_tablet_manager.cc:1428] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:16:10.219314 1313 ts_tablet_manager.cc:1428] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:16:10.219383 994 heartbeater.cc:503] Master 127.30.255.190:33469 was elected leader, sending a full tablet report...
I20260812 11:16:10.219609 1271 heartbeater.cc:503] Master 127.30.255.190:33469 was elected leader, sending a full tablet report...
I20260812 11:16:10.219945 1317 ts_tablet_manager.cc:1428] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260812 11:16:10.243680 728 tablet.cc:2369] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:10.303236 1329 raft_consensus.cc:482] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:10.303447 1329 raft_consensus.cc:504] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.304641 1329 leader_election.cc:288] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010c49306c9e4bb3b24069b6710e653d (127.30.255.129:34987), d0ff1e1735154b4984f644478969a085 (127.30.255.131:34767)
I20260812 11:16:10.308836 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.304929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:35247 (local address 127.30.255.129:34987)
0812 11:16:10.305071 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.305076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.305091 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:16:10.305222 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.305226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.305279 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.305368 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:10.305646 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.306212 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.307687 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.308298 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.308423 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.308478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.308634 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.308712 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:16:10.308836 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.305095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:49299 (local address 127.30.255.131:34767)
0812 11:16:10.305214 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.305218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.305361 (+ 143us) server_negotiation.cc:407] Connection header received
0812 11:16:10.305465 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.305468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.305513 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.305592 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:10.305831 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.306336 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.307782 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.308367 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.308464 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.308504 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.308667 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.308733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:16:10.308845 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.304812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:34987 (local address 127.30.255.130:35247)
0812 11:16:10.304988 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.305003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.305091 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.305388 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.305391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.305409 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:10.305543 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.305550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.306326 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.306329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.307571 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.307582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.308403 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.308405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.308511 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.308537 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.308616 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.308689 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 11:16:10.308857 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.304877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:34767 (local address 127.30.255.130:49299)
0812 11:16:10.305283 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:10.305296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:10.305388 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:10.305598 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:10.305600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:10.305616 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:10.305735 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.305741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.306427 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.306430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.307674 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:10.307682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.308466 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.308469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:10.308554 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.308573 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:16:10.308644 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 11:16:10.308693 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":119,"threads_started":1}
I20260812 11:16:10.309482 627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bd56a06f642b89727069cd7405842" candidate_uuid: "f99c08067d444b6c92e70141989ab814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010c49306c9e4bb3b24069b6710e653d" is_pre_election: true
I20260812 11:16:10.309484 1171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bd56a06f642b89727069cd7405842" candidate_uuid: "f99c08067d444b6c92e70141989ab814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ff1e1735154b4984f644478969a085" is_pre_election: true
I20260812 11:16:10.309777 627 raft_consensus.cc:2454] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99c08067d444b6c92e70141989ab814 in term 0.
I20260812 11:16:10.309777 1171 raft_consensus.cc:2454] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99c08067d444b6c92e70141989ab814 in term 0.
I20260812 11:16:10.310228 767 leader_election.cc:302] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 010c49306c9e4bb3b24069b6710e653d, d0ff1e1735154b4984f644478969a085, f99c08067d444b6c92e70141989ab814; no voters:
I20260812 11:16:10.310448 1329 raft_consensus.cc:2781] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:10.310523 1329 raft_consensus.cc:482] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:10.310573 1329 raft_consensus.cc:3037] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:10.311622 1329 raft_consensus.cc:504] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.311913 1329 leader_election.cc:288] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [CANDIDATE]: Term 1 election: Requested vote from peers 010c49306c9e4bb3b24069b6710e653d (127.30.255.129:34987), d0ff1e1735154b4984f644478969a085 (127.30.255.131:34767)
I20260812 11:16:10.312264 627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bd56a06f642b89727069cd7405842" candidate_uuid: "f99c08067d444b6c92e70141989ab814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010c49306c9e4bb3b24069b6710e653d"
I20260812 11:16:10.312252 1171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bd56a06f642b89727069cd7405842" candidate_uuid: "f99c08067d444b6c92e70141989ab814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ff1e1735154b4984f644478969a085"
I20260812 11:16:10.312378 1171 raft_consensus.cc:3037] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:10.312378 627 raft_consensus.cc:3037] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:10.313797 627 raft_consensus.cc:2454] T 335bd56a06f642b89727069cd7405842 P 010c49306c9e4bb3b24069b6710e653d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99c08067d444b6c92e70141989ab814 in term 1.
I20260812 11:16:10.313874 1171 raft_consensus.cc:2454] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99c08067d444b6c92e70141989ab814 in term 1.
I20260812 11:16:10.314127 767 leader_election.cc:302] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0ff1e1735154b4984f644478969a085, f99c08067d444b6c92e70141989ab814; no voters:
I20260812 11:16:10.314327 1329 raft_consensus.cc:2781] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:10.314497 1329 raft_consensus.cc:686] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [term 1 LEADER]: Becoming Leader. State: Replica: f99c08067d444b6c92e70141989ab814, State: Running, Role: LEADER
I20260812 11:16:10.314817 1329 consensus_queue.cc:237] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } }
I20260812 11:16:10.317863 367 catalog_manager.cc:5495] T 335bd56a06f642b89727069cd7405842 P f99c08067d444b6c92e70141989ab814 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99c08067d444b6c92e70141989ab814 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "f99c08067d444b6c92e70141989ab814" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010c49306c9e4bb3b24069b6710e653d" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99c08067d444b6c92e70141989ab814" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0ff1e1735154b4984f644478969a085" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 34767 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:10.345602 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.337799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.30.255.129:34987)
0812 11:16:10.337929 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.337933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.338077 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:16:10.338175 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.338177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.338216 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.338279 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:10.338883 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.339387 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.340158 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.340358 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.340421 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.342751 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:10.342764 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:10.342777 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:10.342801 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:10.344193 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.344546 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.344551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.344552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.344645 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.344828 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.344830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.344831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.345043 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:10.345187 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.345345 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.345438 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:10.371325 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.363824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.30.255.129:34987)
0812 11:16:10.363950 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.363955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.364080 (+ 125us) server_negotiation.cc:407] Connection header received
0812 11:16:10.364207 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.364209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.364247 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.364307 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:10.364926 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.365427 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.366264 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.366484 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.366570 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.368730 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:10.368733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:10.368735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:10.368757 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:10.369808 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.370181 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.370184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.370185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.370257 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.370495 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.370499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.370500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.370649 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:10.370768 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.371038 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.371144 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[32742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:10.394058 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:10.386957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.30.255.129:34987)
0812 11:16:10.387068 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:10.387072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:10.387231 (+ 159us) server_negotiation.cc:407] Connection header received
0812 11:16:10.387373 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:10.387376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:10.387417 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:10.387475 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:10.388081 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.388590 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.389356 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:10.389571 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:10.389659 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:10.391638 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:10.391642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:10.391644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:10.391666 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:10.392687 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.393040 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.393043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.393045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.393117 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:10.393346 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:10.393349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:10.393351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:10.393496 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:10.393610 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:10.393815 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 11:16:10.393915 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 11:16:10.395334 557 tablet_service.cc:2908] Rejecting scan request for tablet 335bd56a06f642b89727069cd7405842: Uninitialized: clean time has not yet been initialized
I20260812 11:16:10.397115 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 469
W20260812 11:16:10.402207 1272 tablet.cc:2369] T 335bd56a06f642b89727069cd7405842 P d0ff1e1735154b4984f644478969a085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:10.410421 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 738
I20260812 11:16:10.428228 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1000
I20260812 11:16:10.462441 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 308
2026-08-12T11:16:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2460 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[1356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[1356](info): set up 2 sockets
Aug 12 11:16:10 dist-test-slave-gjrz krb5kdc[1356](info): commencing operation
krb5kdc: starting...
W20260812 11:16:11.646530 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:11 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:11.757853 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38219
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:38219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:11.868003 1383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:11.868224 1383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:11.868276 1383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:11.872149 1383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:11.872195 1383 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:16:11.872216 1383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:11.872241 1383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:11.872270 1383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:11.872292 1383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:11.877246 1383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:38219
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:11.878122 1383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:11.879019 1383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:11.886924 1401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:11.886996 1398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:11.886966 1397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:11.887070 1383 server_base.cc:1034] running on GCE node
I20260812 11:16:11.887676 1383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:11.888677 1383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:11.889849 1383 hybrid_clock.cc:648] HybridClock initialized: now 1786533371889823 us; error 33 us; skew 500 ppm
Aug 12 11:16:11 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533371, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:11.897306 1383 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:11.898162 1383 webserver.cc:460] Webserver started at http://127.30.255.190:44685/ using document root <none> and password file <none>
I20260812 11:16:11.898656 1383 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:11.898718 1383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:11.898891 1383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:11.900794 1383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "497bd32655134ecebb139f54ceeefef7"
format_stamp: "Formatted at 2026-08-12 11:16:11 on dist-test-slave-gjrz"
server_key: "ac2d73c1dc51d6a6dfab8fc88cc3363c"
server_key_iv: "b921f30bd1d68b3462b0815c2c9e75c1"
server_key_version: "encryptionkey@0"
I20260812 11:16:11.901311 1383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "497bd32655134ecebb139f54ceeefef7"
format_stamp: "Formatted at 2026-08-12 11:16:11 on dist-test-slave-gjrz"
server_key: "ac2d73c1dc51d6a6dfab8fc88cc3363c"
server_key_iv: "b921f30bd1d68b3462b0815c2c9e75c1"
server_key_version: "encryptionkey@0"
I20260812 11:16:11.904763 1383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 11:16:11.906816 1423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:11.907912 1383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:11.908015 1383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "497bd32655134ecebb139f54ceeefef7"
format_stamp: "Formatted at 2026-08-12 11:16:11 on dist-test-slave-gjrz"
server_key: "ac2d73c1dc51d6a6dfab8fc88cc3363c"
server_key_iv: "b921f30bd1d68b3462b0815c2c9e75c1"
server_key_version: "encryptionkey@0"
I20260812 11:16:11.908104 1383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:11.924790 1383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:11.927134 1383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:11.927328 1383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:11.934621 1383 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:38219
I20260812 11:16:11.934630 1554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:38219 every 8 connection(s)
I20260812 11:16:11.935447 1383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:11.938004 1558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:11.942842 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1383
I20260812 11:16:11.942942 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:11.943159 31742 external_mini_cluster.cc:1428] Setting key 860759ebf67bfc8cf581a5e2a6e91c16
I20260812 11:16:11.943178 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Bootstrap starting.
I20260812 11:16:11.945662 1558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:11.946473 1558 log.cc:824] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:11.948354 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: No bootstrap required, opened a new log
Aug 12 11:16:11 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:11.952260 1558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } }
I20260812 11:16:11.952486 1558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:11.952538 1558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497bd32655134ecebb139f54ceeefef7, State: Initialized, Role: FOLLOWER
I20260812 11:16:11.952955 1558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } }
I20260812 11:16:11.953114 1558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:11.953166 1558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:11.953254 1558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:11.954285 1558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } }
I20260812 11:16:11.954622 1558 leader_election.cc:302] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 497bd32655134ecebb139f54ceeefef7; no voters:
I20260812 11:16:11.954941 1558 leader_election.cc:288] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:11.955024 1569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:11.955207 1569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [term 1 LEADER]: Becoming Leader. State: Replica: 497bd32655134ecebb139f54ceeefef7, State: Running, Role: LEADER
I20260812 11:16:11.955571 1569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } }
I20260812 11:16:11.956146 1558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:11.956741 1573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 497bd32655134ecebb139f54ceeefef7. Latest consensus state: current_term: 1 leader_uuid: "497bd32655134ecebb139f54ceeefef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } } }
I20260812 11:16:11.956874 1573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:11.957214 1571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "497bd32655134ecebb139f54ceeefef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497bd32655134ecebb139f54ceeefef7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38219 } } }
I20260812 11:16:11.957305 1571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:11.957433 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:11.944473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47172 (local address 127.30.255.190:38219)
0812 11:16:11.944870 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:11.944885 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:11.944920 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:16:11.945539 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:11.945556 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:11.945905 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:11.946161 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:11.947033 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:11.948054 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:11.949190 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:11.949643 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:11.949751 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:11.952284 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:11.952289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:11.952301 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:11.952325 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:11.954165 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:11.954578 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:11.954584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:11.954589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:11.954735 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:11.954999 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:11.955002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:11.955004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:11.955546 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:11.955722 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:11.956234 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 11:16:11.956469 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
W20260812 11:16:11.960963 1598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:11.961027 1598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:11.961153 1580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:11.962100 1580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:11.967067 1580 catalog_manager.cc:1347] Generated new cluster ID: 356fbb0a3c04450f95304d810584f3c5
I20260812 11:16:11.967129 1580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:11.974071 1580 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:11.974946 1580 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:11.978334 1580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Generated new TSK 0
I20260812 11:16:11.978842 1580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:12.007081 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38219
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:12.114126 1614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:12.114342 1614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:12.114385 1614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:12.118242 1614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:12.118291 1614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:12.118335 1614 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:12.118386 1614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:12.123346 1614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:12.124225 1614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:12.125129 1614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:12.133908 1627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:12.133911 1625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:12.134016 1614 server_base.cc:1034] running on GCE node
W20260812 11:16:12.133911 1630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:12.134464 1614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:12.134976 1614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:12.136132 1614 hybrid_clock.cc:648] HybridClock initialized: now 1786533372136117 us; error 24 us; skew 500 ppm
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:12.138561 1614 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:12.139371 1614 webserver.cc:460] Webserver started at http://127.30.255.129:37441/ using document root <none> and password file <none>
I20260812 11:16:12.139840 1614 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:12.139894 1614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:12.140043 1614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:12.141819 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "b16603d404f04d5891456b7f56941ff9"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "d8941f96fb40e878c0c079c2724898f1"
server_key_iv: "c2387157b932d25cb9e0ccae5fbb245d"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.142372 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "b16603d404f04d5891456b7f56941ff9"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "d8941f96fb40e878c0c079c2724898f1"
server_key_iv: "c2387157b932d25cb9e0ccae5fbb245d"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.145658 1614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:16:12.147807 1647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.148887 1614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:12.148979 1614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "b16603d404f04d5891456b7f56941ff9"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "d8941f96fb40e878c0c079c2724898f1"
server_key_iv: "c2387157b932d25cb9e0ccae5fbb245d"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.149070 1614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:12.160559 1614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:12.162897 1614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:12.163046 1614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:12.164362 1614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:12.164407 1614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.164443 1614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:12.164469 1614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:12.174762 1614 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:45991
I20260812 11:16:12.174856 1882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:45991 every 8 connection(s)
I20260812 11:16:12.175628 1614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:12.177634 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.165019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:48847 (local address 127.30.255.190:38219)
0812 11:16:12.165135 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.165139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.166069 (+ 930us) server_negotiation.cc:407] Connection header received
0812 11:16:12.166886 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.166890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.166950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.167020 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.168242 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.168769 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.170412 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.170783 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.170895 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.174049 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.174053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.174056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.174081 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.175601 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.176171 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.176175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.176177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.176277 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.176593 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.176597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.176599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.176779 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.176867 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.177318 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.177458 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:12.178283 1667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.165472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.129:48847)
0812 11:16:12.165927 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.165964 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.166694 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.167197 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.167205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.167648 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.168096 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.168109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.168887 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.168891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.170275 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.170288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.170947 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.170951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.171107 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.171590 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.171605 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.173881 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.175713 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.175718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.175731 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.176050 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.176365 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.176369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.176371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.176493 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.176877 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.176882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.177096 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.177251 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.177532 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260812 11:16:12.179558 1884 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38219
I20260812 11:16:12.179688 1884 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:12.180126 1884 heartbeater.cc:511] Master 127.30.255.190:38219 requested a full tablet report, sending...
I20260812 11:16:12.181466 1453 ts_manager.cc:194] Registered new tserver with Master: b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991)
I20260812 11:16:12.182150 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1614
I20260812 11:16:12.182271 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:12.182565 31742 external_mini_cluster.cc:1428] Setting key f2be35bcd16ac252eaea53e85862b2db
I20260812 11:16:12.182819 1453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:48847
I20260812 11:16:12.188493 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.181607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:40269 (local address 127.30.255.190:38219)
0812 11:16:12.181727 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.181732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.181903 (+ 171us) server_negotiation.cc:407] Connection header received
0812 11:16:12.182010 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.182012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.182055 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.182134 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.182598 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.183306 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.184370 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.184598 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.184681 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.185612 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.185616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.185619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.185642 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.187067 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.187523 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.187526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.187527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.187608 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.187853 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.187854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.187855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.187988 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.188074 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.188300 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.188389 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":44}
I20260812 11:16:12.188540 1908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.181530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.129:40269)
0812 11:16:12.181827 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.181837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.181936 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.182161 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.182164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.182350 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.182467 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.182474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.183491 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.183495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.184261 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.184271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.184693 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.184696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.184784 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.185182 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.185195 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.185479 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.187168 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.187170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.187172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.187424 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.187683 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.187686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.187688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.187778 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.188061 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.188064 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.188157 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.188254 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.188374 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:12.209630 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38219
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:12.316318 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:12.316522 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:12.316565 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:12.320407 1915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:12.320453 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:12.320497 1915 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:12.320547 1915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:12.325439 1915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:12.326285 1915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:12.327163 1915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:16:12.336071 1915 server_base.cc:1034] running on GCE node
W20260812 11:16:12.336071 1931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:12.336143 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:12.336220 1933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:12.336621 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:12.337177 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:12.338325 1915 hybrid_clock.cc:648] HybridClock initialized: now 1786533372338308 us; error 26 us; skew 500 ppm
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:12.340797 1915 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:12.341614 1915 webserver.cc:460] Webserver started at http://127.30.255.130:44689/ using document root <none> and password file <none>
I20260812 11:16:12.342106 1915 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:12.342161 1915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:12.342315 1915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:12.344187 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "46875df2a3924c46b44b8b78f7d2ea95"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "e11e7216cc10636b60110ffbe1b72c38"
server_key_iv: "cdf024cb37a5d2f8996e627ce43ba692"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.344746 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "46875df2a3924c46b44b8b78f7d2ea95"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "e11e7216cc10636b60110ffbe1b72c38"
server_key_iv: "cdf024cb37a5d2f8996e627ce43ba692"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.348189 1915 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:12.350394 1944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.351526 1915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:12.351626 1915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "46875df2a3924c46b44b8b78f7d2ea95"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "e11e7216cc10636b60110ffbe1b72c38"
server_key_iv: "cdf024cb37a5d2f8996e627ce43ba692"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.351711 1915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:12.366717 1915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:12.369306 1915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:12.369462 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:12.370755 1915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:12.370800 1915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.370837 1915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:12.370865 1915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:12.380460 1915 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:42239
I20260812 11:16:12.380561 2163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:42239 every 8 connection(s)
I20260812 11:16:12.381429 1915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:12.384006 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.371574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:39391 (local address 127.30.255.190:38219)
0812 11:16:12.371693 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.371698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.372665 (+ 967us) server_negotiation.cc:407] Connection header received
0812 11:16:12.373582 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.373585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.373643 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.373700 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.375372 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.376179 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.377826 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.378173 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.378282 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.380927 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.380931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.380934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.380962 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.382372 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.382804 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.382807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.382808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.382883 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.383154 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.383155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.383156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.383344 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.383416 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.383803 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.383891 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:16:12.384752 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.372049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.130:39391)
0812 11:16:12.372529 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.372568 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.373336 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.374146 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.374155 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.374562 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.375201 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.375220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.376293 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.376296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.377691 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.377704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.378315 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.378318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.378465 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.378960 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.378981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.380767 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.382468 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.382472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.382482 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.382713 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.382965 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.382968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.382969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.383095 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.383419 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.383424 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.383626 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.383771 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.384013 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":129,"threads_started":1}
I20260812 11:16:12.384934 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1915
I20260812 11:16:12.385038 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:12.385326 31742 external_mini_cluster.cc:1428] Setting key cb34583ce63a49414a3b25d1cb9d0612
I20260812 11:16:12.386102 2167 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38219
I20260812 11:16:12.386240 2167 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:12.387399 2167 heartbeater.cc:511] Master 127.30.255.190:38219 requested a full tablet report, sending...
I20260812 11:16:12.388387 1453 ts_manager.cc:194] Registered new tserver with Master: 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130:42239)
I20260812 11:16:12.389076 1453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:39391
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:12.395859 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.388807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.130:59517)
0812 11:16:12.389073 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.389086 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.389228 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.389456 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.389459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.389656 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.389824 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.389832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.390753 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.390756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.391379 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.391391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.391936 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.391939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.392013 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.392425 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.392435 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.392725 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.394321 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.394323 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.394325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.394510 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.394769 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.394771 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.394773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.394833 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.395382 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.395386 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.395492 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.395593 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.395695 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260812 11:16:12.395922 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.389116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:59517 (local address 127.30.255.190:38219)
0812 11:16:12.389210 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.389214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.389228 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:12.389317 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.389319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.389360 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.389425 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.389944 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.390615 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.391499 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.391835 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.391936 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.392856 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.392860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.392862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.392888 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.394222 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.394606 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.394609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.394611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.394693 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.394906 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.394909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.394911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.395199 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.395344 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.395675 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.395779 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:12.413465 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38219
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:12.519735 2186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:12.519949 2186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:12.519987 2186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:12.523792 2186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:12.523836 2186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:12.523871 2186 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:12.523905 2186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:12.528733 2186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:12.529561 2186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:12.530411 2186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:12.539186 2196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:12.539250 2193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:12.539214 2198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:12.539387 2186 server_base.cc:1034] running on GCE node
I20260812 11:16:12.539849 2186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:12.540366 2186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:12.541527 2186 hybrid_clock.cc:648] HybridClock initialized: now 1786533372541506 us; error 33 us; skew 500 ppm
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:12.544081 2186 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:12.544808 2186 webserver.cc:460] Webserver started at http://127.30.255.131:44719/ using document root <none> and password file <none>
I20260812 11:16:12.545284 2186 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:12.545339 2186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:12.545490 2186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:12.547315 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "b741a055ddcc493bbb221afc764bee5b"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "de725328b60f97c1ceaef1baede1bed9"
server_key_iv: "fbfeb130b48bae567fa070a35c02b927"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.547811 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "b741a055ddcc493bbb221afc764bee5b"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "de725328b60f97c1ceaef1baede1bed9"
server_key_iv: "fbfeb130b48bae567fa070a35c02b927"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.551040 2186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:16:12.553236 2206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.554288 2186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:12.554376 2186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "b741a055ddcc493bbb221afc764bee5b"
format_stamp: "Formatted at 2026-08-12 11:16:12 on dist-test-slave-gjrz"
server_key: "de725328b60f97c1ceaef1baede1bed9"
server_key_iv: "fbfeb130b48bae567fa070a35c02b927"
server_key_version: "encryptionkey@0"
I20260812 11:16:12.554450 2186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:12.565701 2186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:12.568363 2186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:12.568492 2186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:12.569681 2186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:12.569729 2186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:12.569780 2186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:12.569811 2186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:12.579847 2186 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:38061
I20260812 11:16:12.580729 2432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:38061 every 8 connection(s)
I20260812 11:16:12.580821 2186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:12.582541 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.570389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:56573 (local address 127.30.255.190:38219)
0812 11:16:12.570529 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.570534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.571310 (+ 776us) server_negotiation.cc:407] Connection header received
0812 11:16:12.572276 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.572280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.572340 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.572396 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.573910 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.574707 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.576138 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.576475 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.576597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.578921 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.578925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.578929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.578957 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.580530 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.581032 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.581037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.581039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.581148 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.581498 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.581502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.581504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.581687 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.581773 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.582269 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.582385 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:16:12.583174 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.570722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.131:56573)
0812 11:16:12.571155 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.571202 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.572029 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.572787 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.572796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.573161 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.573733 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.573755 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.574833 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.574837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.576021 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.576032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.576647 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.576652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.576831 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.577320 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.577335 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.578772 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.580637 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.580641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.580652 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.580937 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.581265 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.581268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.581270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.581418 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.581786 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.581792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.582044 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.582205 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.582474 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260812 11:16:12.584304 2434 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38219
I20260812 11:16:12.584453 2434 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:12.584959 2434 heartbeater.cc:511] Master 127.30.255.190:38219 requested a full tablet report, sending...
I20260812 11:16:12.585932 1452 ts_manager.cc:194] Registered new tserver with Master: b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061)
I20260812 11:16:12.586645 1452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:56573
I20260812 11:16:12.588575 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2186
I20260812 11:16:12.588670 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:12.588932 31742 external_mini_cluster.cc:1428] Setting key f45879029c25bdebe484db90c7cb94f3
I20260812 11:16:12.590951 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:16:12.592973 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.586479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:42199 (local address 127.30.255.190:38219)
0812 11:16:12.586582 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.586587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.586751 (+ 164us) server_negotiation.cc:407] Connection header received
0812 11:16:12.586871 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.586874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.586914 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.586974 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.587427 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.587910 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.588762 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.589061 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.589162 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.590142 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.590145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.590148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.590173 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.591609 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.592037 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.592040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.592041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.592124 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.592362 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.592364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.592365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.592501 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.592564 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.592790 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.592874 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:16:12.593063 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.586384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.30.255.131:42199)
0812 11:16:12.586680 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.586691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.586777 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.586999 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.587003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.587174 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:12.587314 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.587322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.588037 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.588040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.588618 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.588627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.589166 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.589168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.589244 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.589644 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:12.589659 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:12.590028 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:12.591733 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.591736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.591738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.591949 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.592202 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:12.592205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:12.592206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:12.592293 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:12.592573 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:12.592576 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:12.592660 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.592757 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.592878 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":136,"threads_started":1}
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:16:12 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:12.615756 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.608706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.30.255.190:38219)
0812 11:16:12.608819 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.608822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.608832 (+ 10us) server_negotiation.cc:407] Connection header received
0812 11:16:12.608861 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.608863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.608897 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.608960 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:12.609619 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.610105 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.610932 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.611173 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.611258 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.613413 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:12.613416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:12.613418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:12.613441 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:12.614482 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.614825 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.614827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.614829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.614892 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:12.615074 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:12.615076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:12.615077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:12.615223 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:12.615364 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.615527 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.615613 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:16:12.618175 1452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:12.621158 1452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:12.634138 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.628766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:45991 (local address 127.0.0.1:51216)
0812 11:16:12.629531 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.629543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.629633 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.630309 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.630315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.630337 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:12.630444 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.630451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.631515 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.631517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.632678 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.632689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.633645 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.633649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.633753 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.633787 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.633887 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.633999 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260812 11:16:12.634150 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.629145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.30.255.129:45991)
0812 11:16:12.629882 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.629895 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.629917 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:16:12.629991 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.629996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.630166 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.630310 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:12.630567 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.631409 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.632823 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.633527 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.633656 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.633787 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.633926 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.634013 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":113,"threads_started":1}
I20260812 11:16:12.634649 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.629312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38061 (local address 127.0.0.1:49644)
0812 11:16:12.629686 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.629699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.629783 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.630477 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.630480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.630496 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:12.630602 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.630608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.631992 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.632017 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.633180 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.633191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.634263 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.634267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.634360 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.634398 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.634443 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.634507 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":53,"thread_start_us":80,"threads_started":1}
I20260812 11:16:12.635155 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.628650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:42239 (local address 127.0.0.1:44462)
0812 11:16:12.628960 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.629010 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.629239 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.630539 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.630542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.630571 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:12.630681 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.630688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.632172 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.632175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.633226 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.633235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.634367 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.634370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.634863 (+ 493us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.634888 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.634969 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.635036 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":451,"thread_start_us":67,"threads_started":1}
I20260812 11:16:12.635181 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.629318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.30.255.130:42239)
0812 11:16:12.630164 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.630174 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.630192 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:16:12.630254 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.630258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.630403 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.630834 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:12.630967 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.632049 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.633350 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.634128 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.634454 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.634580 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.634998 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.635076 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":107,"threads_started":1}
I20260812 11:16:12.635782 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.629721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.30.255.131:38061)
0812 11:16:12.630006 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.630020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.630041 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:16:12.630128 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.630133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.630317 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.630507 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:12.630706 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.631874 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.633310 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.634119 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.635395 (+ 1276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.635515 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.635573 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.635653 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260812 11:16:12.637207 1761 tablet_service.cc:1467] Processing CreateTablet for tablet 9762de472c7042c2a82ef713f1c05ebc (DEFAULT_TABLE table=test-table [id=63d2ea2c303a4b41a9db3d87071bb934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:12.637619 2029 tablet_service.cc:1467] Processing CreateTablet for tablet 9762de472c7042c2a82ef713f1c05ebc (DEFAULT_TABLE table=test-table [id=63d2ea2c303a4b41a9db3d87071bb934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:12.638176 1761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9762de472c7042c2a82ef713f1c05ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:12.638623 2029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9762de472c7042c2a82ef713f1c05ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:12.638244 2293 tablet_service.cc:1467] Processing CreateTablet for tablet 9762de472c7042c2a82ef713f1c05ebc (DEFAULT_TABLE table=test-table [id=63d2ea2c303a4b41a9db3d87071bb934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:12.639169 2293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9762de472c7042c2a82ef713f1c05ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:12.644510 2479 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Bootstrap starting.
I20260812 11:16:12.646097 2481 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Bootstrap starting.
I20260812 11:16:12.646991 2483 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Bootstrap starting.
I20260812 11:16:12.647002 2479 tablet_bootstrap.cc:654] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:12.647871 2479 log.cc:824] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:12.648152 2481 tablet_bootstrap.cc:654] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:12.648726 2483 tablet_bootstrap.cc:654] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:12.648911 2481 log.cc:824] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:12.649482 2483 log.cc:824] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:12.649880 2479 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: No bootstrap required, opened a new log
I20260812 11:16:12.650051 2479 ts_tablet_manager.cc:1397] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 11:16:12.650661 2481 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: No bootstrap required, opened a new log
I20260812 11:16:12.651033 2481 ts_tablet_manager.cc:1397] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:16:12.651432 2483 tablet_bootstrap.cc:492] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: No bootstrap required, opened a new log
I20260812 11:16:12.651594 2483 ts_tablet_manager.cc:1397] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:16:12.653800 2479 raft_consensus.cc:348] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.654052 2479 raft_consensus.cc:374] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:12.654107 2479 raft_consensus.cc:729] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16603d404f04d5891456b7f56941ff9, State: Initialized, Role: FOLLOWER
I20260812 11:16:12.654106 2481 raft_consensus.cc:348] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.654325 2481 raft_consensus.cc:374] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:12.654384 2481 raft_consensus.cc:729] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b741a055ddcc493bbb221afc764bee5b, State: Initialized, Role: FOLLOWER
I20260812 11:16:12.654524 2479 consensus_queue.cc:260] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.654776 2481 consensus_queue.cc:260] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.655208 1884 heartbeater.cc:503] Master 127.30.255.190:38219 was elected leader, sending a full tablet report...
I20260812 11:16:12.655364 2483 raft_consensus.cc:348] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.655594 2483 raft_consensus.cc:374] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:12.655647 2483 raft_consensus.cc:729] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46875df2a3924c46b44b8b78f7d2ea95, State: Initialized, Role: FOLLOWER
I20260812 11:16:12.655697 2481 ts_tablet_manager.cc:1428] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:16:12.656105 2483 consensus_queue.cc:260] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.656219 2434 heartbeater.cc:503] Master 127.30.255.190:38219 was elected leader, sending a full tablet report...
I20260812 11:16:12.656711 2167 heartbeater.cc:503] Master 127.30.255.190:38219 was elected leader, sending a full tablet report...
I20260812 11:16:12.657389 2479 ts_tablet_manager.cc:1428] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260812 11:16:12.657665 2483 ts_tablet_manager.cc:1428] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 11:16:12.676651 1892 tablet.cc:2369] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:12.831786 2437 tablet.cc:2369] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:12.882542 2171 tablet.cc:2369] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:12.929749 2493 raft_consensus.cc:482] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:12.929955 2493 raft_consensus.cc:504] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.931151 2493 leader_election.cc:288] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061)
I20260812 11:16:12.934953 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.931474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:39175 (local address 127.30.255.129:45991)
0812 11:16:12.931598 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.931603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.931773 (+ 170us) server_negotiation.cc:407] Connection header received
0812 11:16:12.931903 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.931906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.931947 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.932008 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:12.932303 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.932801 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.933938 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.934348 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.934460 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.934508 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.934781 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.934855 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:16:12.934952 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.931409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38061 (local address 127.30.255.130:59213)
0812 11:16:12.931739 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.931752 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.931853 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.932064 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.932067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.932115 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:12.932227 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.932232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.932926 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.932929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.933800 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.933809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.934411 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.934415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.934612 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.934638 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.934717 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.934802 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":20,"spinlock_wait_cycles":768,"thread_start_us":65,"threads_started":1}
I20260812 11:16:12.934955 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.931610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:59213 (local address 127.30.255.131:38061)
0812 11:16:12.931743 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.931747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.931817 (+ 70us) server_negotiation.cc:407] Connection header received
0812 11:16:12.931947 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.931949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.931989 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.932048 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:12.932325 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.932828 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.933914 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.934309 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.934411 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.934468 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.934754 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.934839 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:16:12.935372 2495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.931350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:45991 (local address 127.30.255.130:39175)
0812 11:16:12.931681 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:12.931695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:12.931805 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:12.932032 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:12.932035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:12.932051 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:12.932201 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.932207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.932905 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.932908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.933823 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:12.933832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.934540 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.934544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:12.934640 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.934659 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:16:12.934717 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:16:12.935143 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":424,"thread_start_us":85,"threads_started":1}
I20260812 11:16:12.935667 2340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9762de472c7042c2a82ef713f1c05ebc" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b741a055ddcc493bbb221afc764bee5b" is_pre_election: true
I20260812 11:16:12.935992 2340 raft_consensus.cc:2454] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 0.
I20260812 11:16:12.936072 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9762de472c7042c2a82ef713f1c05ebc" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9" is_pre_election: true
I20260812 11:16:12.936324 1798 raft_consensus.cc:2454] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 0.
I20260812 11:16:12.936465 1949 leader_election.cc:302] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b741a055ddcc493bbb221afc764bee5b; no voters:
I20260812 11:16:12.936762 2493 raft_consensus.cc:2781] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:12.936847 2493 raft_consensus.cc:482] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:12.936896 2493 raft_consensus.cc:3037] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:12.937860 2493 raft_consensus.cc:504] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.938179 2493 leader_election.cc:288] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 election: Requested vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061)
I20260812 11:16:12.938469 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9762de472c7042c2a82ef713f1c05ebc" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9"
I20260812 11:16:12.938498 2340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9762de472c7042c2a82ef713f1c05ebc" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b741a055ddcc493bbb221afc764bee5b"
I20260812 11:16:12.938584 1798 raft_consensus.cc:3037] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:12.938594 2340 raft_consensus.cc:3037] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:12.939836 1798 raft_consensus.cc:2454] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 1.
I20260812 11:16:12.940138 1950 leader_election.cc:302] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b16603d404f04d5891456b7f56941ff9; no voters:
I20260812 11:16:12.940160 2340 raft_consensus.cc:2454] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 1.
I20260812 11:16:12.940323 2493 raft_consensus.cc:2781] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:12.940454 2493 raft_consensus.cc:686] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 LEADER]: Becoming Leader. State: Replica: 46875df2a3924c46b44b8b78f7d2ea95, State: Running, Role: LEADER
I20260812 11:16:12.940759 2493 consensus_queue.cc:237] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:12.943732 1452 catalog_manager.cc:5495] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "46875df2a3924c46b44b8b78f7d2ea95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:12.997619 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:12.994116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.30.255.130:42239)
0812 11:16:12.994241 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:12.994245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:12.994258 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:12.994470 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:12.994473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:12.994522 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:12.994603 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:12.994878 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.995393 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.996245 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:12.996462 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:12.996548 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:12.996876 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:12.997240 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:12.997333 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:12.997447 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 11:16:12.997498 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:16:13.002784 2340 raft_consensus.cc:1261] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Refusing update from remote peer 46875df2a3924c46b44b8b78f7d2ea95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.002784 1798 raft_consensus.cc:1261] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Refusing update from remote peer 46875df2a3924c46b44b8b78f7d2ea95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.003448 2496 consensus_queue.cc:1035] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.003644 2493 consensus_queue.cc:1035] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.021194 1452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:13.021363 1452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:13.023809 1452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Sending DeleteTablet for 3 replicas of tablet 9762de472c7042c2a82ef713f1c05ebc
I20260812 11:16:13.024573 1761 tablet_service.cc:1514] Processing DeleteTablet for tablet 9762de472c7042c2a82ef713f1c05ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:51216
I20260812 11:16:13.024595 2029 tablet_service.cc:1514] Processing DeleteTablet for tablet 9762de472c7042c2a82ef713f1c05ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:44462
I20260812 11:16:13.024817 2293 tablet_service.cc:1514] Processing DeleteTablet for tablet 9762de472c7042c2a82ef713f1c05ebc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:49644
I20260812 11:16:13.025172 2525 tablet_replica.cc:330] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: stopping tablet replica
I20260812 11:16:13.025238 2528 tablet_replica.cc:330] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: stopping tablet replica
I20260812 11:16:13.025470 2525 raft_consensus.cc:2229] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:13.025651 2522 tablet_replica.cc:330] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: stopping tablet replica
I20260812 11:16:13.025780 2525 raft_consensus.cc:2258] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.025754 1452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:16:13.026098 2528 raft_consensus.cc:2229] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 11:16:13.026168 1452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:13.026209 2522 raft_consensus.cc:2229] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:13.026379 2528 raft_consensus.cc:2258] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.026691 2522 raft_consensus.cc:2258] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.027884 2525 ts_tablet_manager.cc:1905] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.027884 2522 ts_tablet_manager.cc:1905] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.027885 2528 ts_tablet_manager.cc:1905] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.030495 2528 ts_tablet_manager.cc:1918] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:13.030524 2525 ts_tablet_manager.cc:1918] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:13.030606 2528 log.cc:1196] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/9762de472c7042c2a82ef713f1c05ebc
I20260812 11:16:13.030615 2525 log.cc:1196] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/9762de472c7042c2a82ef713f1c05ebc
I20260812 11:16:13.030522 2522 ts_tablet_manager.cc:1918] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:13.030843 2522 log.cc:1196] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/9762de472c7042c2a82ef713f1c05ebc
I20260812 11:16:13.030905 2525 ts_tablet_manager.cc:1939] T 9762de472c7042c2a82ef713f1c05ebc P b16603d404f04d5891456b7f56941ff9: Deleting consensus metadata
I20260812 11:16:13.030926 2528 ts_tablet_manager.cc:1939] T 9762de472c7042c2a82ef713f1c05ebc P b741a055ddcc493bbb221afc764bee5b: Deleting consensus metadata
I20260812 11:16:13.031241 2522 ts_tablet_manager.cc:1939] T 9762de472c7042c2a82ef713f1c05ebc P 46875df2a3924c46b44b8b78f7d2ea95: Deleting consensus metadata
I20260812 11:16:13.031726 1761 tablet_service.cc:1467] Processing CreateTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 (DEFAULT_TABLE table=test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:13.031983 1761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1785401ced124c0cb5fe38886b6e02e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.032102 1430 catalog_manager.cc:4841] TS b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991): tablet 9762de472c7042c2a82ef713f1c05ebc (table test-table [id=63d2ea2c303a4b41a9db3d87071bb934]) successfully deleted
I20260812 11:16:13.032297 1429 catalog_manager.cc:4841] TS b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061): tablet 9762de472c7042c2a82ef713f1c05ebc (table test-table [id=63d2ea2c303a4b41a9db3d87071bb934]) successfully deleted
I20260812 11:16:13.032456 1429 catalog_manager.cc:4841] TS 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130:42239): tablet 9762de472c7042c2a82ef713f1c05ebc (table test-table [id=63d2ea2c303a4b41a9db3d87071bb934]) successfully deleted
I20260812 11:16:13.032721 2029 tablet_service.cc:1467] Processing CreateTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 (DEFAULT_TABLE table=test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:13.032873 2293 tablet_service.cc:1467] Processing CreateTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 (DEFAULT_TABLE table=test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:13.032948 2029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1785401ced124c0cb5fe38886b6e02e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.033102 2293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1785401ced124c0cb5fe38886b6e02e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.034160 2479 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Bootstrap starting.
I20260812 11:16:13.035094 2479 tablet_bootstrap.cc:654] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.035136 2483 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Bootstrap starting.
I20260812 11:16:13.035220 2481 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Bootstrap starting.
I20260812 11:16:13.036125 2481 tablet_bootstrap.cc:654] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.036319 2483 tablet_bootstrap.cc:654] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.036741 2479 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: No bootstrap required, opened a new log
I20260812 11:16:13.036821 2479 ts_tablet_manager.cc:1397] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:16:13.037562 2479 raft_consensus.cc:348] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.037680 2479 raft_consensus.cc:374] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.037719 2483 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: No bootstrap required, opened a new log
I20260812 11:16:13.037719 2479 raft_consensus.cc:729] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16603d404f04d5891456b7f56941ff9, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.037786 2483 ts_tablet_manager.cc:1397] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:13.037835 2479 consensus_queue.cc:260] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.038103 2479 ts_tablet_manager.cc:1428] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:13.038182 2481 tablet_bootstrap.cc:492] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: No bootstrap required, opened a new log
I20260812 11:16:13.038250 2481 ts_tablet_manager.cc:1397] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:13.038256 2483 raft_consensus.cc:348] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.038354 2483 raft_consensus.cc:374] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.038384 2483 raft_consensus.cc:729] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46875df2a3924c46b44b8b78f7d2ea95, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.038488 2483 consensus_queue.cc:260] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.038631 2481 raft_consensus.cc:348] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.038728 2481 raft_consensus.cc:374] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.038734 2483 ts_tablet_manager.cc:1428] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:13.038757 2481 raft_consensus.cc:729] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b741a055ddcc493bbb221afc764bee5b, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.038862 2481 consensus_queue.cc:260] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.039088 2481 ts_tablet_manager.cc:1428] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:13.260743 2517 raft_consensus.cc:482] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:13.260874 2517 raft_consensus.cc:504] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.261286 2517 leader_election.cc:288] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061)
I20260812 11:16:13.261678 2340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1785401ced124c0cb5fe38886b6e02e1" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b741a055ddcc493bbb221afc764bee5b" is_pre_election: true
I20260812 11:16:13.261691 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1785401ced124c0cb5fe38886b6e02e1" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9" is_pre_election: true
I20260812 11:16:13.261823 2340 raft_consensus.cc:2454] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 0.
I20260812 11:16:13.261826 1798 raft_consensus.cc:2454] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 0.
I20260812 11:16:13.262138 1950 leader_election.cc:302] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b16603d404f04d5891456b7f56941ff9; no voters:
I20260812 11:16:13.262308 2517 raft_consensus.cc:2781] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:13.262375 2517 raft_consensus.cc:482] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:13.262419 2517 raft_consensus.cc:3037] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.263427 2517 raft_consensus.cc:504] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.263729 2517 leader_election.cc:288] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 election: Requested vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061)
I20260812 11:16:13.264035 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1785401ced124c0cb5fe38886b6e02e1" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9"
I20260812 11:16:13.264066 2340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1785401ced124c0cb5fe38886b6e02e1" candidate_uuid: "46875df2a3924c46b44b8b78f7d2ea95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b741a055ddcc493bbb221afc764bee5b"
I20260812 11:16:13.264147 1798 raft_consensus.cc:3037] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.264158 2340 raft_consensus.cc:3037] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.264967 2340 raft_consensus.cc:2454] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 1.
I20260812 11:16:13.265020 1798 raft_consensus.cc:2454] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46875df2a3924c46b44b8b78f7d2ea95 in term 1.
I20260812 11:16:13.265213 1949 leader_election.cc:302] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b741a055ddcc493bbb221afc764bee5b; no voters:
I20260812 11:16:13.265363 2517 raft_consensus.cc:2781] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:13.265451 2517 raft_consensus.cc:686] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 LEADER]: Becoming Leader. State: Replica: 46875df2a3924c46b44b8b78f7d2ea95, State: Running, Role: LEADER
I20260812 11:16:13.265583 2517 consensus_queue.cc:237] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.267128 1453 catalog_manager.cc:5495] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "46875df2a3924c46b44b8b78f7d2ea95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:16:13 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533371, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:13.312672 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.304827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38219 (local address 127.0.0.1:47182)
0812 11:16:13.305108 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.305121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.305218 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.305435 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.305437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.305820 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:13.305915 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.305920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.306632 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.306634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.307189 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.307196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.307653 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.307656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.307756 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.308310 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:13.308322 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:13.309642 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:13.311107 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:13.311117 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:13.311124 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:13.311406 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:13.311685 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:13.311689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:13.311691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:13.311768 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:13.312117 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:13.312124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:13.312229 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.312324 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.312477 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260812 11:16:13.312672 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.304880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.30.255.190:38219)
0812 11:16:13.305103 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.305110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.305181 (+ 71us) server_negotiation.cc:407] Connection header received
0812 11:16:13.305303 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.305305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.305343 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.305411 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:13.305998 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.306554 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.307310 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.307541 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.307639 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.309763 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:13.309767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:13.309769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:13.309789 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:13.310992 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.311500 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.311502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.311503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.311590 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.311844 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.311847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.311849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.312012 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:13.312097 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.312382 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.312528 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260812 11:16:13.316390 1447 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47182:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:16:13.317174 1447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:13.321674 2029 tablet_service.cc:1467] Processing CreateTablet for tablet e7cd153efc394061a480b8589a1e2afe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e421a2e351144adbcfe17df56a1f120]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:13.321674 2293 tablet_service.cc:1467] Processing CreateTablet for tablet e7cd153efc394061a480b8589a1e2afe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e421a2e351144adbcfe17df56a1f120]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:13.321666 1761 tablet_service.cc:1467] Processing CreateTablet for tablet e7cd153efc394061a480b8589a1e2afe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e421a2e351144adbcfe17df56a1f120]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:13.322079 2029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7cd153efc394061a480b8589a1e2afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.322083 2293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7cd153efc394061a480b8589a1e2afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.322079 1761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7cd153efc394061a480b8589a1e2afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:13.324654 2479 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: Bootstrap starting.
I20260812 11:16:13.324913 2481 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: Bootstrap starting.
I20260812 11:16:13.325583 2479 tablet_bootstrap.cc:654] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.325798 2481 tablet_bootstrap.cc:654] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.326301 2483 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: Bootstrap starting.
I20260812 11:16:13.327018 2481 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: No bootstrap required, opened a new log
I20260812 11:16:13.327104 2481 ts_tablet_manager.cc:1397] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:13.327123 2483 tablet_bootstrap.cc:654] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:13.327567 2481 raft_consensus.cc:348] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.327696 2481 raft_consensus.cc:374] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.327737 2481 raft_consensus.cc:729] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b741a055ddcc493bbb221afc764bee5b, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.327862 2481 consensus_queue.cc:260] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.327973 2479 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: No bootstrap required, opened a new log
I20260812 11:16:13.328050 2479 ts_tablet_manager.cc:1397] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:16:13.328186 2492 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.328317 2492 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.328451 2479 raft_consensus.cc:348] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.328559 2479 raft_consensus.cc:374] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.328593 2479 raft_consensus.cc:729] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16603d404f04d5891456b7f56941ff9, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.328709 2479 consensus_queue.cc:260] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.328785 2481 ts_tablet_manager.cc:1428] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:13.328963 2479 ts_tablet_manager.cc:1428] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:16:13.328943 2491 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.329061 2491 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.329943 2483 tablet_bootstrap.cc:492] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: No bootstrap required, opened a new log
I20260812 11:16:13.330030 2483 ts_tablet_manager.cc:1397] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 11:16:13.330406 2483 raft_consensus.cc:348] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.330500 2483 raft_consensus.cc:374] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:13.330531 2483 raft_consensus.cc:729] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46875df2a3924c46b44b8b78f7d2ea95, State: Initialized, Role: FOLLOWER
I20260812 11:16:13.330663 2483 consensus_queue.cc:260] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.330912 2483 ts_tablet_manager.cc:1428] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:16:13.330907 2517 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.331066 2517 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.380528 2492 raft_consensus.cc:482] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:13.380668 2492 raft_consensus.cc:504] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.381836 2492 leader_election.cc:288] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130:42239)
I20260812 11:16:13.386514 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.382105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:40079 (local address 127.30.255.130:42239)
0812 11:16:13.382302 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.382307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.382705 (+ 398us) server_negotiation.cc:407] Connection header received
0812 11:16:13.382830 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.382833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.382876 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.382941 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:13.383205 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.383991 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.385369 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.385973 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.386090 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.386140 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.386350 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.386420 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 11:16:13.386523 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.382187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:41239 (local address 127.30.255.129:45991)
0812 11:16:13.382302 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.382307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.382516 (+ 209us) server_negotiation.cc:407] Connection header received
0812 11:16:13.382697 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.382700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.382748 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.382811 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:13.383075 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.383746 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.385394 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.385835 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.385908 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.385949 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.386307 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.386407 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:13.386555 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.382010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:45991 (local address 127.30.255.131:41239)
0812 11:16:13.382440 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.382454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.382592 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.382845 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.382848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.382860 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:13.382976 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.382982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.383840 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.383844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.385250 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.385262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.386045 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.386048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.386153 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.386198 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.386274 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.386343 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":54,"thread_start_us":95,"threads_started":1}
I20260812 11:16:13.386881 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.382011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:42239 (local address 127.30.255.131:40079)
0812 11:16:13.382621 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.382635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.382737 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.382955 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.382958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.382975 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:13.383091 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.383097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.384107 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.384111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.385281 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.385288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.386077 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.386088 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.386221 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.386240 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.386648 (+ 408us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.386716 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":270,"thread_start_us":197,"threads_started":1}
I20260812 11:16:13.387318 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7cd153efc394061a480b8589a1e2afe" candidate_uuid: "b741a055ddcc493bbb221afc764bee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9" is_pre_election: true
I20260812 11:16:13.387383 2066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7cd153efc394061a480b8589a1e2afe" candidate_uuid: "b741a055ddcc493bbb221afc764bee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46875df2a3924c46b44b8b78f7d2ea95" is_pre_election: true
I20260812 11:16:13.387462 1798 raft_consensus.cc:2454] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b741a055ddcc493bbb221afc764bee5b in term 0.
I20260812 11:16:13.387584 2066 raft_consensus.cc:2454] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b741a055ddcc493bbb221afc764bee5b in term 0.
I20260812 11:16:13.387943 2214 leader_election.cc:302] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b741a055ddcc493bbb221afc764bee5b; no voters:
I20260812 11:16:13.388163 2492 raft_consensus.cc:2781] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:13.388222 2492 raft_consensus.cc:482] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:13.388264 2492 raft_consensus.cc:3037] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.389183 2492 raft_consensus.cc:504] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.389480 2492 leader_election.cc:288] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [CANDIDATE]: Term 1 election: Requested vote from peers b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991), 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130:42239)
I20260812 11:16:13.389799 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7cd153efc394061a480b8589a1e2afe" candidate_uuid: "b741a055ddcc493bbb221afc764bee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16603d404f04d5891456b7f56941ff9"
I20260812 11:16:13.389812 2066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7cd153efc394061a480b8589a1e2afe" candidate_uuid: "b741a055ddcc493bbb221afc764bee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46875df2a3924c46b44b8b78f7d2ea95"
I20260812 11:16:13.389912 2066 raft_consensus.cc:3037] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.389914 1798 raft_consensus.cc:3037] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:13.390708 1798 raft_consensus.cc:2454] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b741a055ddcc493bbb221afc764bee5b in term 1.
I20260812 11:16:13.390964 2066 raft_consensus.cc:2454] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b741a055ddcc493bbb221afc764bee5b in term 1.
I20260812 11:16:13.391224 2214 leader_election.cc:302] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46875df2a3924c46b44b8b78f7d2ea95, b741a055ddcc493bbb221afc764bee5b; no voters:
I20260812 11:16:13.391402 2492 raft_consensus.cc:2781] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:13.391559 2492 raft_consensus.cc:686] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [term 1 LEADER]: Becoming Leader. State: Replica: b741a055ddcc493bbb221afc764bee5b, State: Running, Role: LEADER
I20260812 11:16:13.391791 2492 consensus_queue.cc:237] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } }
I20260812 11:16:13.393119 2567 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: TxnStatusTablet state changed. Reason: New leader b741a055ddcc493bbb221afc764bee5b. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.393282 2567 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:13.393641 2570 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:16:13.393815 2570 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:16:13.394109 1447 catalog_manager.cc:5495] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b reported cstate change: term changed from 0 to 1, leader changed from <none> to b741a055ddcc493bbb221afc764bee5b (127.30.255.131). New cstate: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:13.405282 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.401914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.30.255.131:38061)
0812 11:16:13.402019 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.402023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.402143 (+ 120us) server_negotiation.cc:407] Connection header received
0812 11:16:13.402313 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.402317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.402364 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.402451 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:13.402683 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.403143 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.403997 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.404204 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.404288 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.404576 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.404934 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.405031 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.405131 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.405176 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:16:13.405313 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.401806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38061 (local address 127.0.0.1:49656)
0812 11:16:13.402066 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.402079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.402189 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.402475 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.402478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.402485 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:13.402585 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.402591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.403245 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.403249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.403873 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.403892 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.404299 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.404303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.404396 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.404440 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.405038 (+ 598us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.405055 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.405118 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.405202 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260812 11:16:13.407791 1798 raft_consensus.cc:1261] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Refusing update from remote peer b741a055ddcc493bbb221afc764bee5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.407971 2066 raft_consensus.cc:1261] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Refusing update from remote peer b741a055ddcc493bbb221afc764bee5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.408340 2492 consensus_queue.cc:1035] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.408502 2567 consensus_queue.cc:1035] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.409848 2491 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: TxnStatusTablet state changed. Reason: New leader b741a055ddcc493bbb221afc764bee5b. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.409953 2491 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.409957 2517 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: TxnStatusTablet state changed. Reason: New leader b741a055ddcc493bbb221afc764bee5b. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.410058 2517 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.411717 2567 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.411814 2567 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:13.411912 2506 mvcc.cc:205] Tried to move back new op lower bound from 7317640697474064384 to 7317640697414795264. Current Snapshot: MvccSnapshot[applied={T|T < 7317640697474064384}]
I20260812 11:16:13.412621 2491 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.412717 2491 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b16603d404f04d5891456b7f56941ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.414911 2492 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.415030 2492 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P b741a055ddcc493bbb221afc764bee5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:13.415529 2517 tablet_replica.cc:439] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b741a055ddcc493bbb221afc764bee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 } } peers { permanent_uuid: "46875df2a3924c46b44b8b78f7d2ea95" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 } } }
I20260812 11:16:13.415639 2517 tablet_replica.cc:442] T e7cd153efc394061a480b8589a1e2afe P 46875df2a3924c46b44b8b78f7d2ea95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:13.426795 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.423757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:42747 (local address 127.30.255.131:38061)
0812 11:16:13.423857 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.423861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.424092 (+ 231us) server_negotiation.cc:407] Connection header received
0812 11:16:13.424204 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.424207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.424249 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.424310 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:13.424544 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.425015 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.425832 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.426059 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.426143 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.426420 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.426487 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.426543 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.426639 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.426683 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:16:13.426832 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.423693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38061 (local address 127.30.255.130:42747)
0812 11:16:13.424014 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.424026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.424122 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.424330 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.424333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.424341 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:13.424448 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.424454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.425122 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.425125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.425721 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.425730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.426162 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.426166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.426253 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.426294 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.426565 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.426572 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.426629 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.426715 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":131,"threads_started":1}
I20260812 11:16:13.432731 1798 raft_consensus.cc:1261] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Refusing update from remote peer 46875df2a3924c46b44b8b78f7d2ea95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.432744 2340 raft_consensus.cc:1261] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Refusing update from remote peer 46875df2a3924c46b44b8b78f7d2ea95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:13.433115 2517 consensus_queue.cc:1035] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b741a055ddcc493bbb221afc764bee5b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.433259 2493 consensus_queue.cc:1035] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16603d404f04d5891456b7f56941ff9" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:13.435894 2505 mvcc.cc:205] Tried to move back new op lower bound from 7317640697577263104 to 7317640696896970752. Current Snapshot: MvccSnapshot[applied={T|T < 7317640697577263104}]
I20260812 11:16:13.438428 2508 mvcc.cc:205] Tried to move back new op lower bound from 7317640697577263104 to 7317640696896970752. Current Snapshot: MvccSnapshot[applied={T|T < 7317640697577263104}]
I20260812 11:16:13.453439 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.449598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:59425 (local address 127.30.255.130:42239)
0812 11:16:13.449698 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.449703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.449919 (+ 216us) server_negotiation.cc:407] Connection header received
0812 11:16:13.450083 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.450086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.450134 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.450192 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:13.450499 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.450973 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.452233 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.452574 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.452655 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.453048 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.453146 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.453209 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.453288 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.453334 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 11:16:13.453691 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.449553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:42239 (local address 127.30.255.131:59425)
0812 11:16:13.449843 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:13.449857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:13.449985 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:13.450222 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:13.450225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:13.450232 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:13.450407 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.450415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.451060 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.451063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.452102 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:13.452114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.452762 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.452766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:13.452860 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.452905 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:13.453213 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:13.453220 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:13.453282 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 11:16:13.453356 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260812 11:16:13.478135 1453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:13.478283 1453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:13.480062 1453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497bd32655134ecebb139f54ceeefef7: Sending DeleteTablet for 3 replicas of tablet 1785401ced124c0cb5fe38886b6e02e1
I20260812 11:16:13.480520 1761 tablet_service.cc:1514] Processing DeleteTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:51216
I20260812 11:16:13.480618 2293 tablet_service.cc:1514] Processing DeleteTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:49644
I20260812 11:16:13.480681 2029 tablet_service.cc:1514] Processing DeleteTablet for tablet 1785401ced124c0cb5fe38886b6e02e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:13 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:44462
I20260812 11:16:13.480777 2528 tablet_replica.cc:330] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: stopping tablet replica
I20260812 11:16:13.480790 2525 tablet_replica.cc:330] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: stopping tablet replica
I20260812 11:16:13.480890 2525 raft_consensus.cc:2229] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:13.480890 2528 raft_consensus.cc:2229] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:13.481002 2528 raft_consensus.cc:2258] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.480999 2525 raft_consensus.cc:2258] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.481172 2522 tablet_replica.cc:330] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: stopping tablet replica
I20260812 11:16:13.481280 2522 raft_consensus.cc:2229] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:13.481438 2522 raft_consensus.cc:2258] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:13.482028 2528 ts_tablet_manager.cc:1905] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.482055 2525 ts_tablet_manager.cc:1905] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.482168 2522 ts_tablet_manager.cc:1905] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:13.484737 2528 ts_tablet_manager.cc:1918] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:13.484738 2525 ts_tablet_manager.cc:1918] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:13.484805 2528 log.cc:1196] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/1785401ced124c0cb5fe38886b6e02e1
I20260812 11:16:13.484805 2525 log.cc:1196] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/1785401ced124c0cb5fe38886b6e02e1
I20260812 11:16:13.484790 2522 ts_tablet_manager.cc:1918] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:13.484856 2522 log.cc:1196] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/1785401ced124c0cb5fe38886b6e02e1
I20260812 11:16:13.485054 2528 ts_tablet_manager.cc:1939] T 1785401ced124c0cb5fe38886b6e02e1 P b741a055ddcc493bbb221afc764bee5b: Deleting consensus metadata
I20260812 11:16:13.485064 2525 ts_tablet_manager.cc:1939] T 1785401ced124c0cb5fe38886b6e02e1 P b16603d404f04d5891456b7f56941ff9: Deleting consensus metadata
I20260812 11:16:13.485107 2522 ts_tablet_manager.cc:1939] T 1785401ced124c0cb5fe38886b6e02e1 P 46875df2a3924c46b44b8b78f7d2ea95: Deleting consensus metadata
I20260812 11:16:13.485767 1430 catalog_manager.cc:4841] TS b16603d404f04d5891456b7f56941ff9 (127.30.255.129:45991): tablet 1785401ced124c0cb5fe38886b6e02e1 (table test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]) successfully deleted
I20260812 11:16:13.485808 1429 catalog_manager.cc:4841] TS b741a055ddcc493bbb221afc764bee5b (127.30.255.131:38061): tablet 1785401ced124c0cb5fe38886b6e02e1 (table test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]) successfully deleted
I20260812 11:16:13.486248 1429 catalog_manager.cc:4841] TS 46875df2a3924c46b44b8b78f7d2ea95 (127.30.255.130:42239): tablet 1785401ced124c0cb5fe38886b6e02e1 (table test-table [id=32b93fb9cf584a2b9c5af5e05d18bcd0]) successfully deleted
Aug 12 11:16:13 dist-test-slave-gjrz krb5kdc[1356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533372, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:13.497881 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.482111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.30.255.129:45991)
0812 11:16:13.482214 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.482219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.487517 (+ 5298us) server_negotiation.cc:407] Connection header received
0812 11:16:13.487632 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.487635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.487680 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.487744 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:13.488417 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.488915 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.489721 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.489947 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.490030 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.493707 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:13.493720 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:13.493738 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:13.493793 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:13.496051 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.496515 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.496523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.496524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.496632 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.496925 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.496929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.496931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.497181 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:13.497368 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.497587 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.497700 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 11:16:13.498258 1873 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51232
I20260812 11:16:13.507294 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:13.500383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.30.255.190:38219)
0812 11:16:13.500499 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:13.500503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:13.500516 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:13.500554 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:13.500557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:13.500595 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:13.500669 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:13.501203 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.501998 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.502702 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:13.503041 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:13.503153 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:13.504323 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:13.504327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:13.504330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:13.504355 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:13.505759 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.506120 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.506123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.506125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.506214 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:13.506464 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:13.506467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:13.506469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:13.506666 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:13.506784 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:13.506994 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 11:16:13.507122 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 11:16:13.507774 1453 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47196
I20260812 11:16:13.508706 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1614
I20260812 11:16:13.551316 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1915
I20260812 11:16:13.562450 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2186
I20260812 11:16:13.578058 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1383
2026-08-12T11:16:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3116 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:13 dist-test-slave-gjrz krb5kdc[2606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:13 dist-test-slave-gjrz krb5kdc[2606](info): set up 2 sockets
Aug 12 11:16:13 dist-test-slave-gjrz krb5kdc[2606](info): commencing operation
krb5kdc: starting...
W20260812 11:16:14.780650 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:14 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:14.891567 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44149
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:44149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:14.997272 2629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:14.997483 2629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:14.997519 2629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:15.001312 2629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:15.001355 2629 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:16:15.001370 2629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:15.001386 2629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:15.001405 2629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:15.001422 2629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:15.006349 2629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:44149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:15.007210 2629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:15.008124 2629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:15.016016 2638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.016041 2639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.016141 2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.016255 2629 server_base.cc:1034] running on GCE node
I20260812 11:16:15.016862 2629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:15.017844 2629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:15.018992 2629 hybrid_clock.cc:648] HybridClock initialized: now 1786533375018968 us; error 42 us; skew 500 ppm
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:15.021479 2629 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:15.022295 2629 webserver.cc:460] Webserver started at http://127.30.255.190:33673/ using document root <none> and password file <none>
I20260812 11:16:15.022776 2629 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:15.022840 2629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:15.023010 2629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:15.024891 2629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "47cb1d53411a4992b619abd635b35ef4"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "a52adb31a8a5ce8388d60ec98419040f"
server_key_iv: "5e5dec2440110e12bbcf4b55bb972aa3"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.025403 2629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "47cb1d53411a4992b619abd635b35ef4"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "a52adb31a8a5ce8388d60ec98419040f"
server_key_iv: "5e5dec2440110e12bbcf4b55bb972aa3"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.028744 2629 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:15.030822 2654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.032033 2629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:15.032163 2629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "47cb1d53411a4992b619abd635b35ef4"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "a52adb31a8a5ce8388d60ec98419040f"
server_key_iv: "5e5dec2440110e12bbcf4b55bb972aa3"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.032253 2629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:15.054735 2629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:15.057616 2629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:15.057781 2629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:15.065208 2629 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:44149
I20260812 11:16:15.065225 2794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:44149 every 8 connection(s)
I20260812 11:16:15.066107 2629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:15.066490 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2629
I20260812 11:16:15.066594 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:15.066859 31742 external_mini_cluster.cc:1428] Setting key 8f00f11b828fe4a9a2fc24e3ae332e25
I20260812 11:16:15.069183 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:15.075390 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Bootstrap starting.
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:15.077926 2797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:15.079182 2797 log.cc:824] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:15.080952 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.068200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.30.255.190:44149)
0812 11:16:15.068605 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.068623 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.068665 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:16:15.069468 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.069491 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.069893 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.070241 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.070852 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.072363 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.073159 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.073820 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.074003 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.076203 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.076210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.076224 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.076254 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.078081 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.078468 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.078473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.078480 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.078632 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.078852 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.078855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.078857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.079371 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.079569 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.079787 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.080073 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260812 11:16:15.081439 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: No bootstrap required, opened a new log
I20260812 11:16:15.084579 2797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } }
I20260812 11:16:15.084805 2797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:15.084867 2797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47cb1d53411a4992b619abd635b35ef4, State: Initialized, Role: FOLLOWER
I20260812 11:16:15.085259 2797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } }
I20260812 11:16:15.085407 2797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:15.085469 2797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:15.085564 2797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:15.086319 2797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } }
I20260812 11:16:15.086627 2797 leader_election.cc:302] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47cb1d53411a4992b619abd635b35ef4; no voters:
I20260812 11:16:15.086902 2797 leader_election.cc:288] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:15.086988 2807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:15.087183 2807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [term 1 LEADER]: Becoming Leader. State: Replica: 47cb1d53411a4992b619abd635b35ef4, State: Running, Role: LEADER
I20260812 11:16:15.087463 2807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } }
I20260812 11:16:15.088009 2797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:15.088969 2809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47cb1d53411a4992b619abd635b35ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } } }
I20260812 11:16:15.089085 2809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:15.089444 2824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:15.090242 2810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47cb1d53411a4992b619abd635b35ef4. Latest consensus state: current_term: 1 leader_uuid: "47cb1d53411a4992b619abd635b35ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cb1d53411a4992b619abd635b35ef4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44149 } } }
I20260812 11:16:15.090353 2810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:15.092553 2824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:15.098191 2824 catalog_manager.cc:1347] Generated new cluster ID: 5667941fdbd649d4b63be99223322fcd
I20260812 11:16:15.098242 2824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:15.105522 2824 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:15.106277 2824 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:15.110782 2824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Generated new TSK 0
I20260812 11:16:15.111356 2824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:15.146281 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44149
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:15.288759 2843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:15.288981 2843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:15.289039 2843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:15.292948 2843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:15.292994 2843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:15.293030 2843 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:16:15.293045 2843 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:15.293078 2843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:15.298054 2843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:15.298928 2843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:15.299847 2843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:15.308722 2850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.308784 2849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.308833 2843 server_base.cc:1034] running on GCE node
W20260812 11:16:15.308732 2854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.309877 2843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:15.310523 2843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:15.311676 2843 hybrid_clock.cc:648] HybridClock initialized: now 1786533375311661 us; error 25 us; skew 500 ppm
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:15.314134 2843 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:15.314889 2843 webserver.cc:460] Webserver started at http://127.30.255.129:44865/ using document root <none> and password file <none>
I20260812 11:16:15.315438 2843 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:15.315505 2843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:15.315685 2843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:15.317531 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "749e5c494ce94c7e9a15b66af1a88f50"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "cec0904fa0f37680e8913d94f71ad572"
server_key_iv: "bc86f8e4af0895f6c20d117b4d83b86b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.318058 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "749e5c494ce94c7e9a15b66af1a88f50"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "cec0904fa0f37680e8913d94f71ad572"
server_key_iv: "bc86f8e4af0895f6c20d117b4d83b86b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.321527 2843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:15.323820 2866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.324950 2843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:15.325053 2843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "749e5c494ce94c7e9a15b66af1a88f50"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "cec0904fa0f37680e8913d94f71ad572"
server_key_iv: "bc86f8e4af0895f6c20d117b4d83b86b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.325155 2843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:15.341486 2843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:15.344013 2843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:15.344177 2843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:15.345420 2843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:15.345466 2843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.345513 2843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:15.345541 2843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:15.355474 2843 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:46565
I20260812 11:16:15.355521 3090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:46565 every 8 connection(s)
I20260812 11:16:15.356452 2843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:15.358371 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.346128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:59615 (local address 127.30.255.190:44149)
0812 11:16:15.346269 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.346274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.347046 (+ 772us) server_negotiation.cc:407] Connection header received
0812 11:16:15.348010 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.348014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.348071 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.348130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.349313 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.350136 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.351714 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.352083 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.352198 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.354804 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.354808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.354810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.354833 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.356391 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.356993 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.356996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.356998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.357097 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.357472 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.357475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.357476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.357655 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.357735 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.358160 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.358257 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:16:15.359089 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.346477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.129:59615)
0812 11:16:15.346896 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.346942 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.347776 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.348467 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.348473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.348761 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.349171 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.349184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.350262 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.350266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.351596 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.351607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.352249 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.352254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.352440 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.352962 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.352977 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.354649 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.356543 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.356549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.356564 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.356820 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.357207 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.357211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.357213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.357378 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.357737 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.357741 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.357969 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.358122 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.358341 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260812 11:16:15.360232 3091 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44149
I20260812 11:16:15.360437 3091 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:15.360888 3091 heartbeater.cc:511] Master 127.30.255.190:44149 requested a full tablet report, sending...
I20260812 11:16:15.362257 2694 ts_manager.cc:194] Registered new tserver with Master: 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565)
I20260812 11:16:15.362490 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2843
I20260812 11:16:15.362607 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:15.362906 31742 external_mini_cluster.cc:1428] Setting key e4eaba658ad95caac2bb17bedd30ff58
I20260812 11:16:15.363590 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:59615
I20260812 11:16:15.368603 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.362250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:46983 (local address 127.30.255.190:44149)
0812 11:16:15.362360 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.362364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.362638 (+ 274us) server_negotiation.cc:407] Connection header received
0812 11:16:15.362801 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.362804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.362848 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.362893 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.363417 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.363974 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.364816 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.365037 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.365118 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.366029 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.366032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.366035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.366059 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.367223 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.367605 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.367608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.367610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.367685 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.367933 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.367936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.367937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.368102 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.368177 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.368416 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.368499 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":49}
I20260812 11:16:15.368662 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.362171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.129:46983)
0812 11:16:15.362557 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.362571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.362702 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.362985 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.362988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.363171 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.363305 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.363314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.364123 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.364127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.364705 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.364714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.365132 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.365136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.365219 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.365609 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.365620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.365897 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.367337 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.367341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.367343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.367520 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.367766 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.367769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.367771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.367854 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.368188 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.368191 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.368277 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.368377 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.368491 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:15.388334 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44149
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:15.497413 3110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:15.497638 3110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:15.497689 3110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:15.501894 3110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:15.501945 3110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:15.501991 3110 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:16:15.502017 3110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:15.502068 3110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:15.507220 3110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:15.508154 3110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:15.509068 3110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:15.517947 3121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.518054 3127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.517997 3123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.518322 3110 server_base.cc:1034] running on GCE node
I20260812 11:16:15.518687 3110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:15.519250 3110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:15.520445 3110 hybrid_clock.cc:648] HybridClock initialized: now 1786533375520428 us; error 25 us; skew 500 ppm
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:15.522912 3110 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:15.523742 3110 webserver.cc:460] Webserver started at http://127.30.255.130:46841/ using document root <none> and password file <none>
I20260812 11:16:15.524227 3110 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:15.524286 3110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:15.524472 3110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:15.526324 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "17567f3dba74af6ce127b24f6c74345f"
server_key_iv: "76d815afd2e6c44e4020e553f19d4c15"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.526839 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "17567f3dba74af6ce127b24f6c74345f"
server_key_iv: "76d815afd2e6c44e4020e553f19d4c15"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.530256 3110 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:15.532509 3137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.533601 3110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:15.533695 3110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "17567f3dba74af6ce127b24f6c74345f"
server_key_iv: "76d815afd2e6c44e4020e553f19d4c15"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.533787 3110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:15.551028 3110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:15.553675 3110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:15.553828 3110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:15.555107 3110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:15.555161 3110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.555198 3110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:15.555227 3110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:15.565182 3110 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:45281
I20260812 11:16:15.565948 3110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:15.566658 3349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:45281 every 8 connection(s)
I20260812 11:16:15.567518 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.555723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:51783 (local address 127.30.255.190:44149)
0812 11:16:15.555866 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.555871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.557070 (+ 1199us) server_negotiation.cc:407] Connection header received
0812 11:16:15.558239 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.558243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.558299 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.558380 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.559667 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.560322 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.561541 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.561847 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.561953 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.564259 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.564263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.564265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.564292 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.565594 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.566032 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.566036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.566037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.566126 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.566468 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.566472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.566473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.566698 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.566793 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.567292 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.567402 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:16:15.568189 3159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.556458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.130:51783)
0812 11:16:15.556893 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.556960 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.557986 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.558549 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.558556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.558961 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.559534 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.559549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.560460 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.560463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.561413 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.561423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.562008 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.562012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.562166 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.562636 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.562650 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.564109 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.565697 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.565702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.565715 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.565943 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.566209 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.566212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.566214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.566357 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.566842 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.566848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.567057 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.567203 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.567459 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260812 11:16:15.569264 3354 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44149
I20260812 11:16:15.569396 3354 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:15.569881 3354 heartbeater.cc:511] Master 127.30.255.190:44149 requested a full tablet report, sending...
I20260812 11:16:15.570863 2697 ts_manager.cc:194] Registered new tserver with Master: 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281)
I20260812 11:16:15.571563 2697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:51783
I20260812 11:16:15.573421 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3110
I20260812 11:16:15.573520 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:15.573792 31742 external_mini_cluster.cc:1428] Setting key 3d7c5517905e8546cb0d9865465e1e75
I20260812 11:16:15.577329 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.571209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.130:41341)
0812 11:16:15.571465 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.571476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.571559 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.571749 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.571752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.571919 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.572003 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.572008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.572680 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.572683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.573238 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.573249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.573751 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.573754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.573846 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.574225 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.574236 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.574559 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.576040 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.576043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.576045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.576262 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.576480 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.576483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.576485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.576567 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.576889 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.576892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.576977 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.577079 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.577212 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":110,"threads_started":1}
I20260812 11:16:15.577379 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.571295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:41341 (local address 127.30.255.190:44149)
0812 11:16:15.571384 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.571388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.571537 (+ 149us) server_negotiation.cc:407] Connection header received
0812 11:16:15.571627 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.571631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.571670 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.571734 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.572104 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.572555 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.573362 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.573636 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.573708 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.574677 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.574680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.574683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.574704 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.575939 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.576342 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.576344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.576345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.576406 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.576635 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.576637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.576638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.576815 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.576883 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.577145 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.577247 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:15.599963 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44149
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:15.707103 3368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:15.707358 3368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:15.707417 3368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:15.711297 3368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:15.711344 3368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:15.711390 3368 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:16:15.711416 3368 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:15.711467 3368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:15.716424 3368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:15.717293 3368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:15.718189 3368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:15.727066 3379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:15.727099 3375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.727170 3368 server_base.cc:1034] running on GCE node
W20260812 11:16:15.727115 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:15.727731 3368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:15.728359 3368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:15.729508 3368 hybrid_clock.cc:648] HybridClock initialized: now 1786533375729490 us; error 29 us; skew 500 ppm
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:15.732096 3368 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:15.732949 3368 webserver.cc:460] Webserver started at http://127.30.255.131:44117/ using document root <none> and password file <none>
I20260812 11:16:15.733471 3368 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:15.733536 3368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:15.733717 3368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:15.735695 3368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "f77c701342704fbbad43f790b3385f86"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "b8b13ce3e8d1dcbfa77b7ee660f86f94"
server_key_iv: "fc8f67a207a15bbdeea0983c25b8819b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.736233 3368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "f77c701342704fbbad43f790b3385f86"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "b8b13ce3e8d1dcbfa77b7ee660f86f94"
server_key_iv: "fc8f67a207a15bbdeea0983c25b8819b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.739773 3368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:15.741971 3393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.743110 3368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:15.743211 3368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "f77c701342704fbbad43f790b3385f86"
format_stamp: "Formatted at 2026-08-12 11:16:15 on dist-test-slave-gjrz"
server_key: "b8b13ce3e8d1dcbfa77b7ee660f86f94"
server_key_iv: "fc8f67a207a15bbdeea0983c25b8819b"
server_key_version: "encryptionkey@0"
I20260812 11:16:15.743333 3368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:15.765067 3368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:15.767858 3368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:15.768014 3368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:15.769312 3368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:15.769362 3368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:15.769411 3368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:15.769436 3368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:15.780709 3368 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:43675
I20260812 11:16:15.780771 3632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:43675 every 8 connection(s)
I20260812 11:16:15.781512 3368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:15.782971 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.769919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35891 (local address 127.30.255.190:44149)
0812 11:16:15.770062 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.770067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.771336 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 11:16:15.772612 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.772615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.772675 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.772751 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.774211 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.774851 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.776390 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.776754 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.776855 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.779641 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.779644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.779647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.779671 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.781067 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.781546 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.781550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.781552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.781648 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.781967 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.781970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.781972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.782160 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.782252 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.782756 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.782856 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:16:15.783670 3405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.770698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.131:35891)
0812 11:16:15.771160 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.771217 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.772330 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.772932 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.772942 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.773397 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.774067 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.774087 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.774975 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.774978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.776274 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.776287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.777026 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.777030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.777224 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.777746 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.777766 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.779474 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.781178 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.781183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.781198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.781442 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.781726 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.781730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.781732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.781858 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.782303 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.782309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.782510 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.782673 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.782895 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260812 11:16:15.784786 3634 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44149
I20260812 11:16:15.784925 3634 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:15.785144 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3368
I20260812 11:16:15.785254 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:15.785393 3634 heartbeater.cc:511] Master 127.30.255.190:44149 requested a full tablet report, sending...
I20260812 11:16:15.785550 31742 external_mini_cluster.cc:1428] Setting key 929b16c9c2fbf6958d5154cc4ad245be
I20260812 11:16:15.786474 2694 ts_manager.cc:194] Registered new tserver with Master: f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:15.787195 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:35891
I20260812 11:16:15.787611 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:16:15.794123 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.786935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.30.255.131:37837)
0812 11:16:15.787656 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.787670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.787795 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.788085 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.788087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.788343 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:15.788463 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.788471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.789394 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.789398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.789934 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.789942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.790461 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.790465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.790562 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.790949 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:15.790963 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:15.791357 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:15.792785 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.792788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.792790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.793021 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.793262 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:15.793265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:15.793267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:15.793357 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:15.793657 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:15.793660 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:15.793752 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.793857 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.793982 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":78,"threads_started":1}
I20260812 11:16:15.794144 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.786972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:37837 (local address 127.30.255.190:44149)
0812 11:16:15.787062 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.787065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.787732 (+ 667us) server_negotiation.cc:407] Connection header received
0812 11:16:15.787926 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.787929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.787972 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.788042 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.788594 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.789256 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.790040 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.790351 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.790426 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.791476 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.791479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.791481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.791500 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.792674 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.793110 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.793112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.793113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.793180 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.793430 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.793432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.793433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.793566 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.793641 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.793904 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.794007 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:16:15 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:15.812866 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.805854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.30.255.190:44149)
0812 11:16:15.805952 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.805955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.805964 (+ 9us) server_negotiation.cc:407] Connection header received
0812 11:16:15.805990 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.805992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.806023 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.806082 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:15.806708 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.807164 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.807987 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.808187 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.808269 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.810290 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:15.810293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:15.810295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:15.810315 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:15.811485 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.811844 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.811847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.811848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.811914 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:15.812157 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:15.812160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:15.812162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:15.812309 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:15.812421 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.812643 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.812738 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:15.815237 2694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:15.818054 2694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:15.831403 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.825587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:45281 (local address 127.0.0.1:56272)
0812 11:16:15.825879 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.825921 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.826029 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.827168 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.827174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.827198 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:15.827339 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.827347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.828531 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.828534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.829892 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.829902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.830868 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.830872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.830975 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.831021 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.831122 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.831234 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":62,"threads_started":1}
I20260812 11:16:15.831419 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.825726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.30.255.130:45281)
0812 11:16:15.826017 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.826028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.826052 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:16:15.826237 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.826244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.826597 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.826973 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:15.827482 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.828437 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.830017 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.830750 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.830874 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.831009 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.831160 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.831255 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
I20260812 11:16:15.831440 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.826161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.0.0.1:50562)
0812 11:16:15.826472 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.826482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.826564 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.827402 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.827405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.827424 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:15.827529 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.827535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.828770 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.828773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.830124 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.830132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.831028 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.831032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.831130 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.831155 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.831204 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.831310 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260812 11:16:15.832150 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.826633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.30.255.131:43675)
0812 11:16:15.826908 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.826920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.826941 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:16:15.827006 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.827011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.827191 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.827426 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:15.827654 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.828661 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.830230 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.830891 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.831778 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.831894 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.831940 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.832022 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260812 11:16:15.832181 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.826226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:46565 (local address 127.0.0.1:42010)
0812 11:16:15.827340 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:15.827353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:15.827572 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:15.828037 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:15.828040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:15.828060 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:15.828160 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.828166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.829378 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.829381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.830382 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:15.830393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.831831 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.831835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:15.831930 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.831951 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:16:15.832009 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:16:15.832079 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"thread_start_us":88,"threads_started":1}
I20260812 11:16:15.832777 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:15.826984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.30.255.129:46565)
0812 11:16:15.827314 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:15.827328 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:15.827550 (+ 222us) server_negotiation.cc:407] Connection header received
0812 11:16:15.827695 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:15.827711 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:15.827902 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:15.828044 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:15.828310 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.829242 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.830508 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:15.831298 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:15.831424 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:15.832524 (+ 1100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:15.832571 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 11:16:15.832651 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
I20260812 11:16:15.834228 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 (DEFAULT_TABLE table=test-table [id=6a94753ddecf4dd5ac7f29658096d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:15.834669 3495 tablet_service.cc:1467] Processing CreateTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 (DEFAULT_TABLE table=test-table [id=6a94753ddecf4dd5ac7f29658096d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:15.835006 2961 tablet_service.cc:1467] Processing CreateTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 (DEFAULT_TABLE table=test-table [id=6a94753ddecf4dd5ac7f29658096d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:15.835386 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f18ad2f8384727b4a1c0629dd76f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:15.835676 3495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f18ad2f8384727b4a1c0629dd76f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:15.835922 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f18ad2f8384727b4a1c0629dd76f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:15.842300 3696 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Bootstrap starting.
I20260812 11:16:15.843546 3698 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Bootstrap starting.
I20260812 11:16:15.843546 3697 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Bootstrap starting.
I20260812 11:16:15.844293 3696 tablet_bootstrap.cc:654] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:15.844970 3696 log.cc:824] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:15.845577 3698 tablet_bootstrap.cc:654] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:15.845577 3697 tablet_bootstrap.cc:654] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:15.846403 3698 log.cc:824] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:15.846403 3697 log.cc:824] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:15.846628 3696 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: No bootstrap required, opened a new log
I20260812 11:16:15.846809 3696 ts_tablet_manager.cc:1397] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 11:16:15.848251 3698 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: No bootstrap required, opened a new log
I20260812 11:16:15.848421 3698 ts_tablet_manager.cc:1397] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260812 11:16:15.848488 3697 tablet_bootstrap.cc:492] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: No bootstrap required, opened a new log
I20260812 11:16:15.848685 3697 ts_tablet_manager.cc:1397] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:16:15.850574 3696 raft_consensus.cc:348] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.850822 3696 raft_consensus.cc:374] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:15.850878 3696 raft_consensus.cc:729] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Initialized, Role: FOLLOWER
I20260812 11:16:15.851145 3698 raft_consensus.cc:348] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.851394 3698 raft_consensus.cc:374] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:15.851336 3696 consensus_queue.cc:260] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.851452 3698 raft_consensus.cc:729] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c701342704fbbad43f790b3385f86, State: Initialized, Role: FOLLOWER
I20260812 11:16:15.851871 3698 consensus_queue.cc:260] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.851964 3091 heartbeater.cc:503] Master 127.30.255.190:44149 was elected leader, sending a full tablet report...
I20260812 11:16:15.852314 3696 ts_tablet_manager.cc:1428] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 11:16:15.852339 3697 raft_consensus.cc:348] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.852567 3697 raft_consensus.cc:374] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:15.852612 3697 raft_consensus.cc:729] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Initialized, Role: FOLLOWER
I20260812 11:16:15.852749 3698 ts_tablet_manager.cc:1428] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260812 11:16:15.853055 3634 heartbeater.cc:503] Master 127.30.255.190:44149 was elected leader, sending a full tablet report...
I20260812 11:16:15.853881 3697 consensus_queue.cc:260] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:15.854791 3697 ts_tablet_manager.cc:1428] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:16:15.854879 3354 heartbeater.cc:503] Master 127.30.255.190:44149 was elected leader, sending a full tablet report...
W20260812 11:16:15.857656 3093 tablet.cc:2369] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:16.032747 3637 tablet.cc:2369] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:16.066972 3355 tablet.cc:2369] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:16.126698 3713 raft_consensus.cc:482] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:16.126910 3713 raft_consensus.cc:504] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.128106 3713 leader_election.cc:288] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:16.131887 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.128444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:53539 (local address 127.30.255.129:46565)
0812 11:16:16.128588 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.128592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.128765 (+ 173us) server_negotiation.cc:407] Connection header received
0812 11:16:16.128908 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.128910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.128950 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.129010 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:16.129334 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.129877 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.130954 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.131371 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.131474 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.131527 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.131699 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.131763 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:16:16.131902 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.128548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:53549 (local address 127.30.255.131:43675)
0812 11:16:16.128674 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.128678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.128827 (+ 149us) server_negotiation.cc:407] Connection header received
0812 11:16:16.128962 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.128965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.129006 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.129066 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:16.129353 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.129849 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.130944 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.131376 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.131474 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.131527 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.131721 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.131785 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:16.131927 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.128381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.30.255.130:53549)
0812 11:16:16.128750 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.128761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.128873 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.129084 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.129088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.129125 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:16.129259 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.129267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.129957 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.129960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.130830 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.130838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.131484 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.131488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.131614 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.131632 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.131695 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.131764 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":65,"spinlock_wait_cycles":768,"thread_start_us":68,"threads_started":1}
I20260812 11:16:16.131933 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.128345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:46565 (local address 127.30.255.130:53539)
0812 11:16:16.128684 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.128694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.128798 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.129037 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.129040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.129061 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:16.129235 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.129241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.129978 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.129982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.130831 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.130840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.131474 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.131477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.131576 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.131611 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.131674 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.131744 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":58,"spinlock_wait_cycles":2432,"thread_start_us":102,"threads_started":1}
I20260812 11:16:16.132540 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f18ad2f8384727b4a1c0629dd76f55" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86" is_pre_election: true
I20260812 11:16:16.132642 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f18ad2f8384727b4a1c0629dd76f55" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50" is_pre_election: true
I20260812 11:16:16.132819 3544 raft_consensus.cc:2454] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:16.132910 2999 raft_consensus.cc:2454] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:16.133286 3140 leader_election.cc:302] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50, f77c701342704fbbad43f790b3385f86; no voters:
I20260812 11:16:16.133538 3713 raft_consensus.cc:2781] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:16.133620 3713 raft_consensus.cc:482] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:16.133663 3713 raft_consensus.cc:3037] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.134658 3713 raft_consensus.cc:504] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.134981 3713 leader_election.cc:288] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Requested vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:16.135327 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f18ad2f8384727b4a1c0629dd76f55" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50"
I20260812 11:16:16.135351 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f18ad2f8384727b4a1c0629dd76f55" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86"
I20260812 11:16:16.135445 3544 raft_consensus.cc:3037] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.135445 2999 raft_consensus.cc:3037] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.136691 2999 raft_consensus.cc:2454] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:16.136981 3144 leader_election.cc:302] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50; no voters:
I20260812 11:16:16.137099 3544 raft_consensus.cc:2454] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:16.137153 3713 raft_consensus.cc:2781] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:16.137346 3713 raft_consensus.cc:686] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Running, Role: LEADER
I20260812 11:16:16.137874 3713 consensus_queue.cc:237] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.140753 2693 catalog_manager.cc:5495] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:16:16 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:16.208678 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.200808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44149 (local address 127.0.0.1:48886)
0812 11:16:16.201112 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.201124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.201231 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.201451 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.201454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.201865 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:16.201993 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.202000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.202720 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.202724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.203300 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.203308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.203748 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.203751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.203873 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.204407 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:16.204418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:16.205847 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:16.207388 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:16.207397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:16.207401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:16.207628 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:16.207843 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:16.207845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:16.207846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:16.207916 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:16.208201 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:16.208205 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:16.208307 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.208391 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.208505 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":18,"thread_start_us":113,"threads_started":1}
I20260812 11:16:16.208681 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.200916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.30.255.190:44149)
0812 11:16:16.201046 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.201051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.201195 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:16:16.201313 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.201317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.201355 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.201443 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:16.202110 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.202621 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.203415 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.203650 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.203737 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.205983 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:16.205986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:16.205988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:16.206011 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:16.207307 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:16.207708 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:16.207710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:16.207711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:16.207784 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:16.207975 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:16.207977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:16.207978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:16.208131 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:16.208199 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.208440 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.208554 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":48}
I20260812 11:16:16.212252 2693 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:48886:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:16:16.212888 2693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:16.217518 3495 tablet_service.cc:1467] Processing CreateTablet for tablet ca518ffefc8b4cedb9030ecab487f32c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5044dd42c20b499589f794a51ff448ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:16.217509 3225 tablet_service.cc:1467] Processing CreateTablet for tablet ca518ffefc8b4cedb9030ecab487f32c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5044dd42c20b499589f794a51ff448ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:16.217509 2961 tablet_service.cc:1467] Processing CreateTablet for tablet ca518ffefc8b4cedb9030ecab487f32c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5044dd42c20b499589f794a51ff448ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:16.218226 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca518ffefc8b4cedb9030ecab487f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.218226 3495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca518ffefc8b4cedb9030ecab487f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.218226 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca518ffefc8b4cedb9030ecab487f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.220878 3696 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: Bootstrap starting.
I20260812 11:16:16.221040 3698 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: Bootstrap starting.
I20260812 11:16:16.221163 3697 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: Bootstrap starting.
I20260812 11:16:16.221725 3696 tablet_bootstrap.cc:654] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.222049 3698 tablet_bootstrap.cc:654] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.222535 3697 tablet_bootstrap.cc:654] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.223757 3698 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: No bootstrap required, opened a new log
I20260812 11:16:16.223798 3697 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: No bootstrap required, opened a new log
I20260812 11:16:16.223841 3698 ts_tablet_manager.cc:1397] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 11:16:16.223874 3697 ts_tablet_manager.cc:1397] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:16.224113 3696 tablet_bootstrap.cc:492] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: No bootstrap required, opened a new log
I20260812 11:16:16.224200 3696 ts_tablet_manager.cc:1397] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 11:16:16.224251 3698 raft_consensus.cc:348] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.224355 3698 raft_consensus.cc:374] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.224313 3697 raft_consensus.cc:348] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.224390 3698 raft_consensus.cc:729] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c701342704fbbad43f790b3385f86, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.224411 3697 raft_consensus.cc:374] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.224442 3697 raft_consensus.cc:729] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.224503 3698 consensus_queue.cc:260] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.224578 3697 consensus_queue.cc:260] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.224627 3696 raft_consensus.cc:348] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.224711 3696 raft_consensus.cc:374] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.224738 3696 raft_consensus.cc:729] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.224761 3698 ts_tablet_manager.cc:1428] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 11:16:16.224843 3697 ts_tablet_manager.cc:1428] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:16.224841 3696 consensus_queue.cc:260] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.225023 3713 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.225087 3696 ts_tablet_manager.cc:1428] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:16.225138 3713 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.225339 3711 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.225463 3711 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.224754 3712 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.225808 3712 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.362622 3712 raft_consensus.cc:482] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:16.362756 3712 raft_consensus.cc:504] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.363981 3712 leader_election.cc:288] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281)
I20260812 11:16:16.367937 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.364183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:46565 (local address 127.30.255.131:34473)
0812 11:16:16.364542 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.364557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.364711 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.365061 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.365064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.365092 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:16.365206 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.365212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.366004 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.366008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.366860 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.366870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.367552 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.367556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.367650 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.367677 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.367740 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.367804 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":80,"threads_started":1}
I20260812 11:16:16.367972 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.364248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:34473 (local address 127.30.255.129:46565)
0812 11:16:16.364360 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.364364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.364636 (+ 272us) server_negotiation.cc:407] Connection header received
0812 11:16:16.364831 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.364834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.364880 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.364920 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:16.365316 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.365896 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.366974 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.367442 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.367533 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.367575 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.367779 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.367853 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:16.368088 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.364845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:37843 (local address 127.30.255.130:45281)
0812 11:16:16.365116 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.365130 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.365143 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:16.365191 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.365194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.365237 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.365321 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:16.365595 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.366152 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.367132 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.367569 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.367682 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.367729 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.367896 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.367966 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
I20260812 11:16:16.368110 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.364565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:45281 (local address 127.30.255.131:37843)
0812 11:16:16.364904 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.364918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.365005 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.365343 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.365346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.365364 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:16.365472 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.365478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.366260 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.366262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.367010 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.367020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.367685 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.367687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.367778 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.367802 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.367871 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.367939 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":72,"threads_started":1}
I20260812 11:16:16.368608 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca518ffefc8b4cedb9030ecab487f32c" candidate_uuid: "f77c701342704fbbad43f790b3385f86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50" is_pre_election: true
I20260812 11:16:16.368616 3274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca518ffefc8b4cedb9030ecab487f32c" candidate_uuid: "f77c701342704fbbad43f790b3385f86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" is_pre_election: true
I20260812 11:16:16.368749 2999 raft_consensus.cc:2454] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f77c701342704fbbad43f790b3385f86 in term 0.
I20260812 11:16:16.368849 3274 raft_consensus.cc:2454] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f77c701342704fbbad43f790b3385f86 in term 0.
I20260812 11:16:16.369055 3397 leader_election.cc:302] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 749e5c494ce94c7e9a15b66af1a88f50, f77c701342704fbbad43f790b3385f86; no voters:
I20260812 11:16:16.369325 3712 raft_consensus.cc:2781] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:16.369374 3712 raft_consensus.cc:482] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:16.369397 3712 raft_consensus.cc:3037] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.370342 3712 raft_consensus.cc:504] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.370638 3712 leader_election.cc:288] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [CANDIDATE]: Term 1 election: Requested vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281)
I20260812 11:16:16.370949 2999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca518ffefc8b4cedb9030ecab487f32c" candidate_uuid: "f77c701342704fbbad43f790b3385f86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50"
I20260812 11:16:16.370988 3274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca518ffefc8b4cedb9030ecab487f32c" candidate_uuid: "f77c701342704fbbad43f790b3385f86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8"
I20260812 11:16:16.371055 2999 raft_consensus.cc:3037] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.371080 3274 raft_consensus.cc:3037] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.371922 2999 raft_consensus.cc:2454] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f77c701342704fbbad43f790b3385f86 in term 1.
I20260812 11:16:16.371984 3274 raft_consensus.cc:2454] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f77c701342704fbbad43f790b3385f86 in term 1.
I20260812 11:16:16.372205 3397 leader_election.cc:302] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 749e5c494ce94c7e9a15b66af1a88f50, f77c701342704fbbad43f790b3385f86; no voters:
I20260812 11:16:16.372394 3712 raft_consensus.cc:2781] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:16.372562 3712 raft_consensus.cc:686] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [term 1 LEADER]: Becoming Leader. State: Replica: f77c701342704fbbad43f790b3385f86, State: Running, Role: LEADER
I20260812 11:16:16.372846 3712 consensus_queue.cc:237] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.374755 3742 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: TxnStatusTablet state changed. Reason: New leader f77c701342704fbbad43f790b3385f86. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.374845 3742 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:16.375089 3746 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:16:16.375293 3746 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:16:16.376017 2693 catalog_manager.cc:5495] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 reported cstate change: term changed from 0 to 1, leader changed from <none> to f77c701342704fbbad43f790b3385f86 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:16.410050 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.406624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.0.0.1:50572)
0812 11:16:16.406780 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.406795 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.406898 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.407208 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.407210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.407218 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:16.407339 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.407345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.408021 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.408025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.408623 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.408631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.409039 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.409042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.409128 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.409173 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.409781 (+ 608us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.409798 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.409861 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.409940 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:16:16.410054 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.406725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.30.255.131:43675)
0812 11:16:16.406829 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.406833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.406846 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:16.407031 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.407034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.407097 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.407176 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:16.407441 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.407917 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.408736 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.408943 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.409027 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.409304 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.409686 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.409781 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.409873 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.409921 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:16.414999 3274 raft_consensus.cc:1261] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Refusing update from remote peer f77c701342704fbbad43f790b3385f86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.415208 2999 raft_consensus.cc:1261] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Refusing update from remote peer f77c701342704fbbad43f790b3385f86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.415560 3742 consensus_queue.cc:1035] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:16.415763 3712 consensus_queue.cc:1035] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:16.418855 3713 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: TxnStatusTablet state changed. Reason: New leader f77c701342704fbbad43f790b3385f86. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.418969 3713 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.419130 3711 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: TxnStatusTablet state changed. Reason: New leader f77c701342704fbbad43f790b3385f86. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.419225 3711 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.423022 3712 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.423111 3712 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:16.423708 3712 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.423827 3712 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P f77c701342704fbbad43f790b3385f86: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:16.425124 3713 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.425225 3713 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 0ac750ac1e984a25bb6d5487a4b0f8a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.425633 3711 tablet_replica.cc:439] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f77c701342704fbbad43f790b3385f86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } } }
I20260812 11:16:16.425745 3711 tablet_replica.cc:442] T ca518ffefc8b4cedb9030ecab487f32c P 749e5c494ce94c7e9a15b66af1a88f50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:16.425706 3757 mvcc.cc:205] Tried to move back new op lower bound from 7317640709790109696 to 7317640709625376768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640709790109696}]
I20260812 11:16:16.425886 3751 mvcc.cc:205] Tried to move back new op lower bound from 7317640709790109696 to 7317640709625376768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640709790109696}]
I20260812 11:16:16.427358 3756 mvcc.cc:205] Tried to move back new op lower bound from 7317640709790109696 to 7317640709625376768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640709790109696 or (T in {7317640709790109696})}]
I20260812 11:16:16.436009 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.432202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56286 (local address 127.30.255.130:45281)
0812 11:16:16.432323 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.432327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.432538 (+ 211us) server_negotiation.cc:407] Connection header received
0812 11:16:16.432674 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.432676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.432721 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.432795 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:16.433011 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.433483 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.434639 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.434876 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.434952 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.435298 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.435674 (+ 376us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.435750 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.435849 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.435894 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:16:16.445863 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.442798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:53645 (local address 127.30.255.131:43675)
0812 11:16:16.442894 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.442898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.443135 (+ 237us) server_negotiation.cc:407] Connection header received
0812 11:16:16.443237 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.443240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.443310 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.443371 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:16.443605 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.444081 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.444911 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.445119 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.445201 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.445483 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.445544 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.445607 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.445699 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.445744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:16:16.445923 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.442724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.30.255.130:53645)
0812 11:16:16.443053 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.443065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.443158 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.443390 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.443392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.443399 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:16.443509 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.443514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.444180 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.444183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.444803 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.444811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.445213 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.445216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.445308 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.445346 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.445625 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.445632 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.445690 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.445762 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":162,"threads_started":1}
I20260812 11:16:16.453122 2999 raft_consensus.cc:1261] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.453122 3544 raft_consensus.cc:1261] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.453526 3713 consensus_queue.cc:1035] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:16.453698 3718 consensus_queue.cc:1035] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:16.475343 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.471918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45253 (local address 127.30.255.130:45281)
0812 11:16:16.472064 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:16.472069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:16.472254 (+ 185us) server_negotiation.cc:407] Connection header received
0812 11:16:16.472428 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:16.472431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:16.472480 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:16.472526 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:16.472899 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.473559 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.474364 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.474589 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.474675 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.474928 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.474998 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.475064 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:16.475150 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 11:16:16.475195 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:16:16.475378 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:16.471830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:45281 (local address 127.30.255.131:45253)
0812 11:16:16.472184 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:16.472199 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:16.472322 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:16.472675 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:16.472677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:16.472684 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:16.472775 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.472781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.473662 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.473666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.474257 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:16.474266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:16.474688 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:16.474691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:16.474774 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:16.474815 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:16.475076 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:16.475083 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:16.475140 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 11:16:16.475216 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
I20260812 11:16:16.496968 2692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:16.497167 2692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:16.499962 2692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Sending DeleteTablet for 3 replicas of tablet 96f18ad2f8384727b4a1c0629dd76f55
I20260812 11:16:16.500741 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:16 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42010
I20260812 11:16:16.500821 3495 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:16 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260812 11:16:16.500864 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f18ad2f8384727b4a1c0629dd76f55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:16 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:56272
I20260812 11:16:16.501413 3798 tablet_replica.cc:330] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: stopping tablet replica
I20260812 11:16:16.501713 3798 raft_consensus.cc:2229] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:16.501891 3798 raft_consensus.cc:2258] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:16.501997 2692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:16.502374 2692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:16.503185 3798 ts_tablet_manager.cc:1905] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:16.507287 3495 tablet_service.cc:1467] Processing CreateTablet for tablet f84c029c86d540f18f19234f7b896460 (DEFAULT_TABLE table=test-table [id=f1172b35a6b14c84a83273d9e8194a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:16.507532 3495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c029c86d540f18f19234f7b896460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.508177 3798 ts_tablet_manager.cc:1918] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:16.508260 3798 log.cc:1196] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/96f18ad2f8384727b4a1c0629dd76f55
I20260812 11:16:16.508562 3798 ts_tablet_manager.cc:1939] T 96f18ad2f8384727b4a1c0629dd76f55 P f77c701342704fbbad43f790b3385f86: Deleting consensus metadata
I20260812 11:16:16.509722 2658 catalog_manager.cc:4841] TS f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675): tablet 96f18ad2f8384727b4a1c0629dd76f55 (table test-table [id=6a94753ddecf4dd5ac7f29658096d768]) successfully deleted
I20260812 11:16:16.509817 3698 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Bootstrap starting.
I20260812 11:16:16.510826 3698 tablet_bootstrap.cc:654] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.511435 3796 tablet_replica.cc:330] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: stopping tablet replica
I20260812 11:16:16.511739 3796 raft_consensus.cc:2229] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:16.511946 3796 raft_consensus.cc:2258] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:16.512024 3698 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: No bootstrap required, opened a new log
I20260812 11:16:16.512094 3698 ts_tablet_manager.cc:1397] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:16.512305 3795 tablet_replica.cc:330] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: stopping tablet replica
I20260812 11:16:16.512470 2961 tablet_service.cc:1467] Processing CreateTablet for tablet f84c029c86d540f18f19234f7b896460 (DEFAULT_TABLE table=test-table [id=f1172b35a6b14c84a83273d9e8194a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:16.512555 3795 raft_consensus.cc:2229] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:16.512487 3698 raft_consensus.cc:348] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.512609 3698 raft_consensus.cc:374] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.512640 3698 raft_consensus.cc:729] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c701342704fbbad43f790b3385f86, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.512696 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c029c86d540f18f19234f7b896460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.512750 3698 consensus_queue.cc:260] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.512879 3795 raft_consensus.cc:2258] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:16.513003 3698 ts_tablet_manager.cc:1428] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:16.514150 3796 ts_tablet_manager.cc:1905] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:16.514886 3696 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Bootstrap starting.
I20260812 11:16:16.515024 3225 tablet_service.cc:1467] Processing CreateTablet for tablet f84c029c86d540f18f19234f7b896460 (DEFAULT_TABLE table=test-table [id=f1172b35a6b14c84a83273d9e8194a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:16.515246 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c029c86d540f18f19234f7b896460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:16.515791 3696 tablet_bootstrap.cc:654] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.516594 3796 ts_tablet_manager.cc:1918] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:16.516672 3796 log.cc:1196] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/96f18ad2f8384727b4a1c0629dd76f55
I20260812 11:16:16.516906 3696 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: No bootstrap required, opened a new log
I20260812 11:16:16.516952 3796 ts_tablet_manager.cc:1939] T 96f18ad2f8384727b4a1c0629dd76f55 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting consensus metadata
I20260812 11:16:16.516978 3696 ts_tablet_manager.cc:1397] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:16.517364 3696 raft_consensus.cc:348] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.517457 3696 raft_consensus.cc:374] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.517486 3696 raft_consensus.cc:729] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.517604 3696 consensus_queue.cc:260] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.517648 3697 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Bootstrap starting.
I20260812 11:16:16.517834 3696 ts_tablet_manager.cc:1428] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:16.518484 3697 tablet_bootstrap.cc:654] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:16.518730 3795 ts_tablet_manager.cc:1905] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:16.519742 2664 catalog_manager.cc:4841] TS 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565): tablet 96f18ad2f8384727b4a1c0629dd76f55 (table test-table [id=6a94753ddecf4dd5ac7f29658096d768]) successfully deleted
I20260812 11:16:16.521420 3795 ts_tablet_manager.cc:1918] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:16.521514 3795 log.cc:1196] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/96f18ad2f8384727b4a1c0629dd76f55
I20260812 11:16:16.521801 3795 ts_tablet_manager.cc:1939] T 96f18ad2f8384727b4a1c0629dd76f55 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting consensus metadata
I20260812 11:16:16.522655 3697 tablet_bootstrap.cc:492] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: No bootstrap required, opened a new log
I20260812 11:16:16.522733 3697 ts_tablet_manager.cc:1397] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 11:16:16.522783 2664 catalog_manager.cc:4841] TS 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281): tablet 96f18ad2f8384727b4a1c0629dd76f55 (table test-table [id=6a94753ddecf4dd5ac7f29658096d768]) successfully deleted
I20260812 11:16:16.523123 3697 raft_consensus.cc:348] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.523228 3697 raft_consensus.cc:374] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:16.523257 3697 raft_consensus.cc:729] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Initialized, Role: FOLLOWER
I20260812 11:16:16.523401 3697 consensus_queue.cc:260] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.523638 3697 ts_tablet_manager.cc:1428] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:16.543838 3713 raft_consensus.cc:482] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:16.543926 3713 raft_consensus.cc:504] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.544263 3713 leader_election.cc:288] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:16.544602 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c029c86d540f18f19234f7b896460" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86" is_pre_election: true
I20260812 11:16:16.544632 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c029c86d540f18f19234f7b896460" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50" is_pre_election: true
I20260812 11:16:16.544726 3544 raft_consensus.cc:2454] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:16.544767 2998 raft_consensus.cc:2454] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:16.544993 3140 leader_election.cc:302] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, f77c701342704fbbad43f790b3385f86; no voters:
I20260812 11:16:16.545167 3713 raft_consensus.cc:2781] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:16.545223 3713 raft_consensus.cc:482] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:16.545269 3713 raft_consensus.cc:3037] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.546142 3713 raft_consensus.cc:504] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.546430 3713 leader_election.cc:288] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Requested vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:16.546768 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c029c86d540f18f19234f7b896460" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50"
I20260812 11:16:16.546794 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c029c86d540f18f19234f7b896460" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86"
I20260812 11:16:16.546887 2998 raft_consensus.cc:3037] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.546902 3544 raft_consensus.cc:3037] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:16.547886 2998 raft_consensus.cc:2454] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:16.547973 3544 raft_consensus.cc:2454] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:16.548174 3144 leader_election.cc:302] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50; no voters:
I20260812 11:16:16.548358 3713 raft_consensus.cc:2781] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:16.548511 3713 raft_consensus.cc:686] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Running, Role: LEADER
I20260812 11:16:16.548646 3713 consensus_queue.cc:237] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:16.550192 2692 catalog_manager.cc:5495] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:16.571126 2998 raft_consensus.cc:1261] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.571688 3718 consensus_queue.cc:1035] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:16.571919 3544 raft_consensus.cc:1261] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:16.572806 3718 consensus_queue.cc:1035] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:16:16.588483 3756 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 11:16:18.139882 2692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:18.140026 2692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:18.142135 2692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Sending DeleteTablet for 3 replicas of tablet f84c029c86d540f18f19234f7b896460
I20260812 11:16:18.142594 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c029c86d540f18f19234f7b896460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42010
I20260812 11:16:18.142623 3495 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c029c86d540f18f19234f7b896460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260812 11:16:18.142679 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c029c86d540f18f19234f7b896460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:56272
I20260812 11:16:18.142921 3866 tablet_replica.cc:330] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: stopping tablet replica
I20260812 11:16:18.143011 3868 tablet_replica.cc:330] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: stopping tablet replica
I20260812 11:16:18.143079 3866 raft_consensus.cc:2229] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:18.143136 3868 raft_consensus.cc:2229] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:18.143186 3866 raft_consensus.cc:2258] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.143329 3868 raft_consensus.cc:2258] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.143419 3867 tablet_replica.cc:330] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: stopping tablet replica
I20260812 11:16:18.143540 3867 raft_consensus.cc:2229] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:18.143642 3867 raft_consensus.cc:2258] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.144241 3868 ts_tablet_manager.cc:1905] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.144622 3867 ts_tablet_manager.cc:1905] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.146566 3866 ts_tablet_manager.cc:1905] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.148633 3868 ts_tablet_manager.cc:1918] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:16:18.148774 3868 log.cc:1196] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/f84c029c86d540f18f19234f7b896460
I20260812 11:16:18.148813 3867 ts_tablet_manager.cc:1918] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:16:18.148955 3867 log.cc:1196] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/f84c029c86d540f18f19234f7b896460
I20260812 11:16:18.149084 3866 ts_tablet_manager.cc:1918] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:16:18.149147 3868 ts_tablet_manager.cc:1939] T f84c029c86d540f18f19234f7b896460 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting consensus metadata
I20260812 11:16:18.149202 3866 log.cc:1196] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/f84c029c86d540f18f19234f7b896460
I20260812 11:16:18.149317 3867 ts_tablet_manager.cc:1939] T f84c029c86d540f18f19234f7b896460 P f77c701342704fbbad43f790b3385f86: Deleting consensus metadata
I20260812 11:16:18.149461 3866 ts_tablet_manager.cc:1939] T f84c029c86d540f18f19234f7b896460 P 749e5c494ce94c7e9a15b66af1a88f50: Deleting consensus metadata
I20260812 11:16:18.149852 2664 catalog_manager.cc:4841] TS 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281): tablet f84c029c86d540f18f19234f7b896460 (table test-table [id=f1172b35a6b14c84a83273d9e8194a1d]) successfully deleted
I20260812 11:16:18.150058 2658 catalog_manager.cc:4841] TS f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675): tablet f84c029c86d540f18f19234f7b896460 (table test-table [id=f1172b35a6b14c84a83273d9e8194a1d]) successfully deleted
I20260812 11:16:18.150130 2664 catalog_manager.cc:4841] TS 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565): tablet f84c029c86d540f18f19234f7b896460 (table test-table [id=f1172b35a6b14c84a83273d9e8194a1d]) successfully deleted
Aug 12 11:16:18 dist-test-slave-gjrz krb5kdc[2606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:16:18 dist-test-slave-gjrz krb5kdc[2606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:18.176174 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.162686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.30.255.190:44149)
0812 11:16:18.162918 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.162927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.162993 (+ 66us) server_negotiation.cc:407] Connection header received
0812 11:16:18.163140 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.163142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.163184 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.163256 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:18.163966 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.164514 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.165322 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.165550 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.165638 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.173017 (+ 7379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:18.173020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:18.173022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:18.173046 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:18.174584 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:18.174975 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:18.174980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:18.174982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:18.175064 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:18.175341 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:18.175345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:18.175346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:18.175541 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:18.175662 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.175901 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.176022 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
I20260812 11:16:18.178408 2692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:18.178787 2692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:18.182886 2961 tablet_service.cc:1467] Processing CreateTablet for tablet 56765486484242e29e9eb1a8180926fb (DEFAULT_TABLE table=test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.182934 3225 tablet_service.cc:1467] Processing CreateTablet for tablet 56765486484242e29e9eb1a8180926fb (DEFAULT_TABLE table=test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.183003 3495 tablet_service.cc:1467] Processing CreateTablet for tablet 56765486484242e29e9eb1a8180926fb (DEFAULT_TABLE table=test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.183185 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56765486484242e29e9eb1a8180926fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.183215 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56765486484242e29e9eb1a8180926fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.183220 3495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56765486484242e29e9eb1a8180926fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.185473 3888 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Bootstrap starting.
I20260812 11:16:18.185606 3889 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Bootstrap starting.
I20260812 11:16:18.186082 3890 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Bootstrap starting.
I20260812 11:16:18.186470 3888 tablet_bootstrap.cc:654] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.186590 3889 tablet_bootstrap.cc:654] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.186887 3890 tablet_bootstrap.cc:654] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.187757 3888 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: No bootstrap required, opened a new log
I20260812 11:16:18.187834 3888 ts_tablet_manager.cc:1397] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:18.188244 3888 raft_consensus.cc:348] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.188380 3888 raft_consensus.cc:374] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.188421 3888 raft_consensus.cc:729] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.188561 3888 consensus_queue.cc:260] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.188627 3890 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: No bootstrap required, opened a new log
I20260812 11:16:18.188705 3890 ts_tablet_manager.cc:1397] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:16:18.188848 3888 ts_tablet_manager.cc:1428] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:18.189095 3890 raft_consensus.cc:348] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.189183 3890 raft_consensus.cc:374] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.189203 3890 raft_consensus.cc:729] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.189324 3890 consensus_queue.cc:260] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.189728 3890 ts_tablet_manager.cc:1428] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:18.189754 3889 tablet_bootstrap.cc:492] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: No bootstrap required, opened a new log
I20260812 11:16:18.189831 3889 ts_tablet_manager.cc:1397] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 11:16:18.190312 3889 raft_consensus.cc:348] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.190416 3889 raft_consensus.cc:374] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.190447 3889 raft_consensus.cc:729] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c701342704fbbad43f790b3385f86, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.190579 3889 consensus_queue.cc:260] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.190865 3889 ts_tablet_manager.cc:1428] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:18.236737 3900 raft_consensus.cc:482] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:18.236881 3900 raft_consensus.cc:504] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.237986 3900 leader_election.cc:288] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:18.243175 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.238271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:48181 (local address 127.30.255.130:45281)
0812 11:16:18.238552 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.238562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.238631 (+ 69us) server_negotiation.cc:407] Connection header received
0812 11:16:18.238791 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.238793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.238846 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.238937 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:18.239340 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.240162 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.241859 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.242496 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.242619 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.242672 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.242965 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.243040 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
I20260812 11:16:18.243175 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.238271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:53381 (local address 127.30.255.131:43675)
0812 11:16:18.238563 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.238572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.238735 (+ 163us) server_negotiation.cc:407] Connection header received
0812 11:16:18.238821 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.238823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.238866 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.238944 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:18.239343 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.240156 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.241859 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.242495 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.242619 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.242672 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.242965 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.243039 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":111,"threads_started":1}
I20260812 11:16:18.243216 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.238197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:45281 (local address 127.30.255.129:48181)
0812 11:16:18.238539 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:18.238554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:18.238665 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:18.238956 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:18.238959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:18.239016 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:18.239207 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.239213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.240272 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.240276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.241737 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.241749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.242645 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.242650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.242820 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.242867 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 11:16:18.242944 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 11:16:18.243031 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":219,"spinlock_wait_cycles":44672,"thread_start_us":107,"threads_started":1}
I20260812 11:16:18.243218 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.238197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.30.255.129:53381)
0812 11:16:18.238653 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:18.238666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:18.238748 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:18.238956 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:18.238959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:18.238988 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:18.239197 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.239206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.240272 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.240276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.241737 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.241749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.242645 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.242650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.242817 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.242860 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 11:16:18.242944 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:16:18.243031 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":162,"spinlock_wait_cycles":16000,"thread_start_us":89,"threads_started":1}
I20260812 11:16:18.243642 3274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56765486484242e29e9eb1a8180926fb" candidate_uuid: "749e5c494ce94c7e9a15b66af1a88f50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" is_pre_election: true
I20260812 11:16:18.243786 3274 raft_consensus.cc:2454] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 749e5c494ce94c7e9a15b66af1a88f50 in term 0.
I20260812 11:16:18.243996 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56765486484242e29e9eb1a8180926fb" candidate_uuid: "749e5c494ce94c7e9a15b66af1a88f50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86" is_pre_election: true
I20260812 11:16:18.244141 3544 raft_consensus.cc:2454] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 749e5c494ce94c7e9a15b66af1a88f50 in term 0.
I20260812 11:16:18.244169 2872 leader_election.cc:302] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50; no voters:
I20260812 11:16:18.244621 3900 raft_consensus.cc:2781] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:18.244678 3900 raft_consensus.cc:482] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:18.244704 3900 raft_consensus.cc:3037] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.246057 3900 raft_consensus.cc:504] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.246366 3900 leader_election.cc:288] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [CANDIDATE]: Term 1 election: Requested vote from peers 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:18.246663 3274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56765486484242e29e9eb1a8180926fb" candidate_uuid: "749e5c494ce94c7e9a15b66af1a88f50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8"
I20260812 11:16:18.246723 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56765486484242e29e9eb1a8180926fb" candidate_uuid: "749e5c494ce94c7e9a15b66af1a88f50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86"
I20260812 11:16:18.246780 3274 raft_consensus.cc:3037] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.246827 3544 raft_consensus.cc:3037] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.247764 3544 raft_consensus.cc:2454] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 749e5c494ce94c7e9a15b66af1a88f50 in term 1.
I20260812 11:16:18.247781 3274 raft_consensus.cc:2454] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 749e5c494ce94c7e9a15b66af1a88f50 in term 1.
I20260812 11:16:18.248021 2868 leader_election.cc:302] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 749e5c494ce94c7e9a15b66af1a88f50, f77c701342704fbbad43f790b3385f86; no voters:
I20260812 11:16:18.248221 3900 raft_consensus.cc:2781] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:18.248407 3900 raft_consensus.cc:686] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 LEADER]: Becoming Leader. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Running, Role: LEADER
I20260812 11:16:18.248651 3900 consensus_queue.cc:237] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.250530 2694 catalog_manager.cc:5495] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "749e5c494ce94c7e9a15b66af1a88f50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:18.272563 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.269032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.30.255.129:46565)
0812 11:16:18.269266 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.269272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.269284 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:18.269344 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.269346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.269393 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.269507 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:18.269802 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.270324 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.271132 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.271417 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.271501 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.271768 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.272201 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.272304 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.272374 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.272422 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260812 11:16:18.281592 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.278629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:44381 (local address 127.30.255.131:43675)
0812 11:16:18.278749 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.278753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.278861 (+ 108us) server_negotiation.cc:407] Connection header received
0812 11:16:18.278979 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.278982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.279026 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.279097 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:18.279338 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.279817 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.280637 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.280853 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.280934 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.281213 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.281277 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.281333 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.281423 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.281472 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:18.281636 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.278502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43675 (local address 127.30.255.129:44381)
0812 11:16:18.278791 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:18.278802 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:18.278894 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:18.279121 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:18.279125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:18.279133 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:18.279231 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.279236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.279917 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.279920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.280517 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.280525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.280966 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.280969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.281046 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.281090 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.281349 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.281356 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:18.281414 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:16:18.281488 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260812 11:16:18.286474 3274 raft_consensus.cc:1261] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Refusing update from remote peer 749e5c494ce94c7e9a15b66af1a88f50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:18.286711 3544 raft_consensus.cc:1261] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Refusing update from remote peer 749e5c494ce94c7e9a15b66af1a88f50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:18.286911 3907 consensus_queue.cc:1035] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:18.287070 3900 consensus_queue.cc:1035] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:18.303766 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.300679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:34769 (local address 127.30.255.129:46565)
0812 11:16:18.300879 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.300883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.300893 (+ 10us) server_negotiation.cc:407] Connection header received
0812 11:16:18.300929 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.300931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.300976 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.301043 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:18.301331 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.301833 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.302706 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.302992 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.303087 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.303346 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.303418 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.303481 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.303625 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.303675 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 11:16:18.303769 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.300439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:46565 (local address 127.30.255.131:34769)
0812 11:16:18.300693 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:18.300707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:18.300797 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:18.301147 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:18.301150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:18.301157 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:18.301238 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.301242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.301924 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.301926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.302603 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:18.302613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.303098 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.303101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:18.303183 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.303215 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.303522 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.303529 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:16:18.303591 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 11:16:18.303661 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260812 11:16:18.324770 2692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:18.324913 2692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:18.326814 2692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Sending DeleteTablet for 3 replicas of tablet 56765486484242e29e9eb1a8180926fb
I20260812 11:16:18.327419 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet 56765486484242e29e9eb1a8180926fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:56272
I20260812 11:16:18.327438 3495 tablet_service.cc:1514] Processing DeleteTablet for tablet 56765486484242e29e9eb1a8180926fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260812 11:16:18.327600 3868 tablet_replica.cc:330] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: stopping tablet replica
I20260812 11:16:18.327706 3868 raft_consensus.cc:2229] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:18.327803 3868 raft_consensus.cc:2258] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.327960 3867 tablet_replica.cc:330] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: stopping tablet replica
I20260812 11:16:18.328099 3867 raft_consensus.cc:2229] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:18.328199 3867 raft_consensus.cc:2258] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.328083 2692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:18.328523 2692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:18.329130 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet 56765486484242e29e9eb1a8180926fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:18 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42010
I20260812 11:16:18.330035 3866 tablet_replica.cc:330] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: stopping tablet replica
I20260812 11:16:18.330156 3866 raft_consensus.cc:2229] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:18.330230 3867 ts_tablet_manager.cc:1905] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.330428 3866 raft_consensus.cc:2258] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:18.331020 3868 ts_tablet_manager.cc:1905] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.332746 3867 ts_tablet_manager.cc:1918] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:18.332767 3866 ts_tablet_manager.cc:1905] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:18.332816 3867 log.cc:1196] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/56765486484242e29e9eb1a8180926fb
I20260812 11:16:18.333035 3867 ts_tablet_manager.cc:1939] T 56765486484242e29e9eb1a8180926fb P f77c701342704fbbad43f790b3385f86: Deleting consensus metadata
I20260812 11:16:18.335525 2658 catalog_manager.cc:4841] TS f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675): tablet 56765486484242e29e9eb1a8180926fb (table test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]) successfully deleted
I20260812 11:16:18.335505 3225 tablet_service.cc:1467] Processing CreateTablet for tablet e53472ef44834e9498c50fa1cdeffe28 (DEFAULT_TABLE table=test-table [id=c09d0a46ea344b89a68688f8fd0f1e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.335628 2961 tablet_service.cc:1467] Processing CreateTablet for tablet e53472ef44834e9498c50fa1cdeffe28 (DEFAULT_TABLE table=test-table [id=c09d0a46ea344b89a68688f8fd0f1e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.335776 3225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53472ef44834e9498c50fa1cdeffe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.335846 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53472ef44834e9498c50fa1cdeffe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.338007 3495 tablet_service.cc:1467] Processing CreateTablet for tablet e53472ef44834e9498c50fa1cdeffe28 (DEFAULT_TABLE table=test-table [id=c09d0a46ea344b89a68688f8fd0f1e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:18.338093 3888 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Bootstrap starting.
I20260812 11:16:18.338233 3495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53472ef44834e9498c50fa1cdeffe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:18.338838 3888 tablet_bootstrap.cc:654] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.339388 3890 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: Bootstrap starting.
I20260812 11:16:18.339931 3888 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: No bootstrap required, opened a new log
I20260812 11:16:18.340030 3888 ts_tablet_manager.cc:1397] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:18.340317 3890 tablet_bootstrap.cc:654] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.340368 3888 raft_consensus.cc:348] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.340477 3888 raft_consensus.cc:374] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.340513 3888 raft_consensus.cc:729] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.340636 3888 consensus_queue.cc:260] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.340878 3888 ts_tablet_manager.cc:1428] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:18.341956 3868 ts_tablet_manager.cc:1918] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:18.342027 3868 log.cc:1196] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/56765486484242e29e9eb1a8180926fb
I20260812 11:16:18.342267 3868 ts_tablet_manager.cc:1939] T 56765486484242e29e9eb1a8180926fb P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting consensus metadata
I20260812 11:16:18.342769 3866 ts_tablet_manager.cc:1918] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:18.342852 3866 log.cc:1196] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/56765486484242e29e9eb1a8180926fb
I20260812 11:16:18.342854 2664 catalog_manager.cc:4841] TS 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281): tablet 56765486484242e29e9eb1a8180926fb (table test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]) successfully deleted
I20260812 11:16:18.343080 3866 ts_tablet_manager.cc:1939] T 56765486484242e29e9eb1a8180926fb P 749e5c494ce94c7e9a15b66af1a88f50: Deleting consensus metadata
I20260812 11:16:18.343644 3889 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Bootstrap starting.
I20260812 11:16:18.344022 2664 catalog_manager.cc:4841] TS 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565): tablet 56765486484242e29e9eb1a8180926fb (table test-table [id=ca1ca0d74f3f4d02aa75d4dd5758be6d]) successfully deleted
I20260812 11:16:18.344385 3889 tablet_bootstrap.cc:654] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:18.344506 3890 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: No bootstrap required, opened a new log
I20260812 11:16:18.344579 3890 ts_tablet_manager.cc:1397] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 11:16:18.344980 3890 raft_consensus.cc:348] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.345095 3890 raft_consensus.cc:374] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.345131 3890 raft_consensus.cc:729] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749e5c494ce94c7e9a15b66af1a88f50, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.345244 3890 consensus_queue.cc:260] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.345507 3890 ts_tablet_manager.cc:1428] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:18.346165 3889 tablet_bootstrap.cc:492] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: No bootstrap required, opened a new log
I20260812 11:16:18.346244 3889 ts_tablet_manager.cc:1397] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 11:16:18.346606 3889 raft_consensus.cc:348] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.346719 3889 raft_consensus.cc:374] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:18.346757 3889 raft_consensus.cc:729] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c701342704fbbad43f790b3385f86, State: Initialized, Role: FOLLOWER
I20260812 11:16:18.346880 3889 consensus_queue.cc:260] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.347115 3889 ts_tablet_manager.cc:1428] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:18.473295 3811 raft_consensus.cc:482] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:18.473428 3811 raft_consensus.cc:504] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.473852 3811 leader_election.cc:288] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:18.474264 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53472ef44834e9498c50fa1cdeffe28" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50" is_pre_election: true
I20260812 11:16:18.474270 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53472ef44834e9498c50fa1cdeffe28" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86" is_pre_election: true
I20260812 11:16:18.474404 2998 raft_consensus.cc:2454] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:18.474413 3544 raft_consensus.cc:2454] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 0.
I20260812 11:16:18.474718 3144 leader_election.cc:302] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50; no voters:
I20260812 11:16:18.474907 3811 raft_consensus.cc:2781] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:18.474993 3811 raft_consensus.cc:482] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:18.475018 3811 raft_consensus.cc:3037] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.476056 3811 raft_consensus.cc:504] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.476363 3811 leader_election.cc:288] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Requested vote from peers 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565), f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675)
I20260812 11:16:18.476701 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53472ef44834e9498c50fa1cdeffe28" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "749e5c494ce94c7e9a15b66af1a88f50"
I20260812 11:16:18.476724 3544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53472ef44834e9498c50fa1cdeffe28" candidate_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77c701342704fbbad43f790b3385f86"
I20260812 11:16:18.476810 2998 raft_consensus.cc:3037] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.476830 3544 raft_consensus.cc:3037] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:18.477660 2998 raft_consensus.cc:2454] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:18.477770 3544 raft_consensus.cc:2454] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac750ac1e984a25bb6d5487a4b0f8a8 in term 1.
I20260812 11:16:18.477916 3144 leader_election.cc:302] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac750ac1e984a25bb6d5487a4b0f8a8, 749e5c494ce94c7e9a15b66af1a88f50; no voters:
I20260812 11:16:18.478084 3811 raft_consensus.cc:2781] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:18.478212 3811 raft_consensus.cc:686] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac750ac1e984a25bb6d5487a4b0f8a8, State: Running, Role: LEADER
I20260812 11:16:18.478345 3811 consensus_queue.cc:237] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } }
I20260812 11:16:18.479957 2694 catalog_manager.cc:5495] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac750ac1e984a25bb6d5487a4b0f8a8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:18.499449 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:18.496476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.30.255.130:45281)
0812 11:16:18.496592 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:18.496597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:18.496634 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:16:18.496777 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:18.496780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:18.496825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:18.496885 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:18.497133 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.497641 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.498447 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:18.498674 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:18.498758 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:18.499010 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:18.499082 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:18.499139 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:18.499243 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 11:16:18.499317 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:18.504640 2998 raft_consensus.cc:1261] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:18.504652 3544 raft_consensus.cc:1261] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Refusing update from remote peer 0ac750ac1e984a25bb6d5487a4b0f8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:18.504968 3811 consensus_queue.cc:1035] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "749e5c494ce94c7e9a15b66af1a88f50" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:18.505143 3846 consensus_queue.cc:1035] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c701342704fbbad43f790b3385f86" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:16:18.516412 3818 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 11:16:20.061074 2694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:20.061228 2694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:20.063390 2694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cb1d53411a4992b619abd635b35ef4: Sending DeleteTablet for 3 replicas of tablet e53472ef44834e9498c50fa1cdeffe28
I20260812 11:16:20.063848 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet e53472ef44834e9498c50fa1cdeffe28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:20 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42010
I20260812 11:16:20.063948 3225 tablet_service.cc:1514] Processing DeleteTablet for tablet e53472ef44834e9498c50fa1cdeffe28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:20 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:56272
I20260812 11:16:20.064141 3495 tablet_service.cc:1514] Processing DeleteTablet for tablet e53472ef44834e9498c50fa1cdeffe28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:20 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50562
I20260812 11:16:20.064273 3985 tablet_replica.cc:330] T e53472ef44834e9498c50fa1cdeffe28 P 749e5c494ce94c7e9a15b66af1a88f50: stopping tablet replica
I20260812 11:16:20.064283 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2843
I20260812 11:16:20.076076 3987 tablet_replica.cc:330] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: stopping tablet replica
I20260812 11:16:20.076223 3987 raft_consensus.cc:2229] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:20.076406 3987 raft_consensus.cc:2258] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:20.076977 3987 ts_tablet_manager.cc:1905] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:20.079444 3988 tablet_replica.cc:330] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: stopping tablet replica
I20260812 11:16:20.079627 3988 raft_consensus.cc:2229] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:20.079780 3988 raft_consensus.cc:2258] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:20.079778 3987 ts_tablet_manager.cc:1918] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:16:20.079849 3987 log.cc:1196] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/e53472ef44834e9498c50fa1cdeffe28
I20260812 11:16:20.080262 3987 ts_tablet_manager.cc:1939] T e53472ef44834e9498c50fa1cdeffe28 P 0ac750ac1e984a25bb6d5487a4b0f8a8: Deleting consensus metadata
I20260812 11:16:20.080461 3988 ts_tablet_manager.cc:1905] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:20.081023 2664 catalog_manager.cc:4841] TS 0ac750ac1e984a25bb6d5487a4b0f8a8 (127.30.255.130:45281): tablet e53472ef44834e9498c50fa1cdeffe28 (table test-table [id=c09d0a46ea344b89a68688f8fd0f1e1c]) successfully deleted
I20260812 11:16:20.083007 3988 ts_tablet_manager.cc:1918] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:16:20.083096 3988 log.cc:1196] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/e53472ef44834e9498c50fa1cdeffe28
I20260812 11:16:20.083385 3988 ts_tablet_manager.cc:1939] T e53472ef44834e9498c50fa1cdeffe28 P f77c701342704fbbad43f790b3385f86: Deleting consensus metadata
I20260812 11:16:20.084101 2658 catalog_manager.cc:4841] TS f77c701342704fbbad43f790b3385f86 (127.30.255.131:43675): tablet e53472ef44834e9498c50fa1cdeffe28 (table test-table [id=c09d0a46ea344b89a68688f8fd0f1e1c]) successfully deleted
I20260812 11:16:20.084726 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3110
W20260812 11:16:20.084758 2664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:46565 (error 108)
I20260812 11:16:20.089721 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:20.088213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:46565 (local address 127.0.0.1:43882)
0812 11:16:20.089577 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:20.089648 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1293,"thread_start_us":1030,"threads_started":1}
W20260812 11:16:20.089846 2664 catalog_manager.cc:4584] TS 749e5c494ce94c7e9a15b66af1a88f50 (127.30.255.129:46565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e53472ef44834e9498c50fa1cdeffe28: Network error: Client connection negotiation failed: client connection to 127.30.255.129:46565: connect: Connection refused (error 111)
I20260812 11:16:20.096742 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3368
I20260812 11:16:20.110373 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2629
2026-08-12T11:16:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6535 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:20 dist-test-slave-gjrz krb5kdc[4000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:20 dist-test-slave-gjrz krb5kdc[4000](info): set up 2 sockets
Aug 12 11:16:20 dist-test-slave-gjrz krb5kdc[4000](info): commencing operation
krb5kdc: starting...
W20260812 11:16:21.312553 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:21.426126 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:35057
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:35057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:21.536880 4026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:21.537103 4026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:21.537140 4026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:21.541075 4026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:21.541122 4026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:21.541141 4026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:21.541154 4026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:21.541179 4026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:21.546180 4026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:35057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:21.547072 4026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:21.548017 4026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:21.556119 4043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:21.556197 4037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:21.556164 4038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:21.556197 4026 server_base.cc:1034] running on GCE node
I20260812 11:16:21.556895 4026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:21.557868 4026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:21.559031 4026 hybrid_clock.cc:648] HybridClock initialized: now 1786533381559017 us; error 25 us; skew 500 ppm
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:21.561635 4026 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:21.562463 4026 webserver.cc:460] Webserver started at http://127.30.255.190:35499/ using document root <none> and password file <none>
I20260812 11:16:21.562951 4026 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:21.563011 4026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:21.563200 4026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:21.565109 4026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "0b3214501b6b4e8e906a4a3f41c9eedb"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "8a41c2845ba949281312e19cd985bbe2"
server_key_iv: "860c30ba8271c5a2b5839fcec3da4cea"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.565629 4026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "0b3214501b6b4e8e906a4a3f41c9eedb"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "8a41c2845ba949281312e19cd985bbe2"
server_key_iv: "860c30ba8271c5a2b5839fcec3da4cea"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.569106 4026 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:21.571277 4056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:21.572418 4026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:21.572532 4026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0b3214501b6b4e8e906a4a3f41c9eedb"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "8a41c2845ba949281312e19cd985bbe2"
server_key_iv: "860c30ba8271c5a2b5839fcec3da4cea"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.572618 4026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:21.582325 4026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:21.585117 4026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:21.585273 4026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:21.591765 4026 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:35057
I20260812 11:16:21.591833 4159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:35057 every 8 connection(s)
I20260812 11:16:21.592628 4026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:21.595124 4160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:21.600267 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb: Bootstrap starting.
I20260812 11:16:21.601003 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4026
I20260812 11:16:21.601099 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:21.601327 31742 external_mini_cluster.cc:1428] Setting key a06be8ae718363023938cbb6f3af91c8
I20260812 11:16:21.602840 4160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:21.603740 4160 log.cc:824] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:21.605739 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb: No bootstrap required, opened a new log
I20260812 11:16:21.609668 4160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } }
I20260812 11:16:21.609930 4160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:21.609992 4160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3214501b6b4e8e906a4a3f41c9eedb, State: Initialized, Role: FOLLOWER
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:21.610442 4160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } }
I20260812 11:16:21.610577 4160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:21.610641 4160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:21.610747 4160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:21.611869 4160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } }
I20260812 11:16:21.612221 4160 leader_election.cc:302] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3214501b6b4e8e906a4a3f41c9eedb; no voters:
I20260812 11:16:21.612568 4160 leader_election.cc:288] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:21.612653 4166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:21.612856 4166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [term 1 LEADER]: Becoming Leader. State: Replica: 0b3214501b6b4e8e906a4a3f41c9eedb, State: Running, Role: LEADER
I20260812 11:16:21.613193 4166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } }
I20260812 11:16:21.613798 4160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:21.614676 4172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b3214501b6b4e8e906a4a3f41c9eedb. Latest consensus state: current_term: 1 leader_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } } }
I20260812 11:16:21.614794 4172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:21.615100 4171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3214501b6b4e8e906a4a3f41c9eedb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 35057 } } }
I20260812 11:16:21.615198 4171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:21.615914 4185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:21.616565 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:21.602398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.30.255.190:35057)
0812 11:16:21.602845 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:21.602864 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:21.602920 (+ 56us) server_negotiation.cc:407] Connection header received
0812 11:16:21.603747 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:21.603772 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:21.604176 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:21.604462 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:21.605112 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.606514 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.607480 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.608197 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.608369 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:21.610748 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:21.610755 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:21.610771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:21.610803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:21.612676 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:21.613216 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:21.613222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:21.613227 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:21.613355 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:21.613659 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:21.613662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:21.613664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:21.614167 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:21.615066 (+ 899us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:21.615220 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 11:16:21.615568 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":118,"threads_started":1}
I20260812 11:16:21.618707 4185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:21.624059 4185 catalog_manager.cc:1347] Generated new cluster ID: f2df03bba4424c64994d51483564b97c
I20260812 11:16:21.624116 4185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:21.633198 4185 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:21.633962 4185 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:21.638796 4185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b3214501b6b4e8e906a4a3f41c9eedb: Generated new TSK 0
I20260812 11:16:21.639369 4185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:21.665997 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:21.772240 4213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:21.772460 4213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:21.772513 4213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:21.776394 4213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:21.776444 4213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:21.776515 4213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:21.781440 4213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:21.782325 4213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:21.783215 4213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:21.792093 4227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:21.792237 4213 server_base.cc:1034] running on GCE node
W20260812 11:16:21.792086 4230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:21.792086 4226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:21.792873 4213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:21.793426 4213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:21.794579 4213 hybrid_clock.cc:648] HybridClock initialized: now 1786533381794558 us; error 29 us; skew 500 ppm
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:21.797757 4213 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:21.798497 4213 webserver.cc:460] Webserver started at http://127.30.255.129:42207/ using document root <none> and password file <none>
I20260812 11:16:21.798996 4213 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:21.799059 4213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:21.799238 4213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:21.801211 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "1d3a719edaf54c89b6be424b3656f3c1"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "9f63ee2ddd2a3385c6355c8a410ac5b0"
server_key_iv: "b7d77ee372291e73317e37afd4f2595d"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.801733 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "1d3a719edaf54c89b6be424b3656f3c1"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "9f63ee2ddd2a3385c6355c8a410ac5b0"
server_key_iv: "b7d77ee372291e73317e37afd4f2595d"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.805150 4213 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 11:16:21.807312 4246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:21.808415 4213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:21.808517 4213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "1d3a719edaf54c89b6be424b3656f3c1"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "9f63ee2ddd2a3385c6355c8a410ac5b0"
server_key_iv: "b7d77ee372291e73317e37afd4f2595d"
server_key_version: "encryptionkey@0"
I20260812 11:16:21.808614 4213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:21.822046 4213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:21.824465 4213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:21.824625 4213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:21.825117 4213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:21.825829 4213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:21.825873 4213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:21.825922 4213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:21.825951 4213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:21.833966 4213 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:35211
I20260812 11:16:21.833989 4457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:35211 every 8 connection(s)
I20260812 11:16:21.834715 4213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:21.841353 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4213
I20260812 11:16:21.841457 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:21.841753 31742 external_mini_cluster.cc:1428] Setting key b549c407f70019afec1f76a06b20ef9a
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:21.847198 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:21.836295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:56745 (local address 127.30.255.190:35057)
0812 11:16:21.836413 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:21.836417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:21.837258 (+ 841us) server_negotiation.cc:407] Connection header received
0812 11:16:21.838445 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:21.838448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:21.838492 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:21.838562 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:21.839612 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.840132 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.841235 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.841569 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.841680 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:21.844029 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:21.844032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:21.844034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:21.844058 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:21.845367 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:21.845817 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:21.845821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:21.845822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:21.845915 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:21.846209 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:21.846212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:21.846214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:21.846418 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:21.846543 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:21.846939 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 11:16:21.847053 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:16:21.847874 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:21.836667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:35057 (local address 127.30.255.129:56745)
0812 11:16:21.837124 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:21.837162 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:21.838255 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:21.838701 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:21.838707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:21.839013 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:21.839495 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:21.839509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.840234 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.840237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:21.841123 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:21.841134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:21.841708 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:21.841711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:21.841846 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:21.842327 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:21.842342 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:21.843874 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:21.845478 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:21.845482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:21.845492 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:21.845731 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:21.845994 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:21.845996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:21.845998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:21.846129 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:21.846505 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:21.846509 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:21.846722 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 11:16:21.846864 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 11:16:21.847094 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
I20260812 11:16:21.849031 4463 heartbeater.cc:347] Connected to a master server at 127.30.255.190:35057
I20260812 11:16:21.849277 4463 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:21.849809 4463 heartbeater.cc:511] Master 127.30.255.190:35057 requested a full tablet report, sending...
I20260812 11:16:21.851230 4091 ts_manager.cc:194] Registered new tserver with Master: 1d3a719edaf54c89b6be424b3656f3c1 (127.30.255.129:35211)
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:21.852347 4091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:56745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:21.867681 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:21.972105 4473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:21.972321 4473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:21.972359 4473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:21.976226 4473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:21.976271 4473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:21.976339 4473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:21.981345 4473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:21.982230 4473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:21.983120 4473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:21.992007 4479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:21.992084 4473 server_base.cc:1034] running on GCE node
W20260812 11:16:21.992025 4481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:21.992017 4485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:21.992599 4473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:21.993134 4473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:21.994282 4473 hybrid_clock.cc:648] HybridClock initialized: now 1786533381994261 us; error 35 us; skew 500 ppm
Aug 12 11:16:21 dist-test-slave-gjrz krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:21.996742 4473 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:21.997498 4473 webserver.cc:460] Webserver started at http://127.30.255.130:40253/ using document root <none> and password file <none>
I20260812 11:16:21.997980 4473 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:21.998044 4473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:21.998217 4473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:22.000077 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "447463d2d30d4d8cb13031da4eff9a62"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "1125745479cdf537334501b2d6f26d40"
server_key_iv: "03c8c1eaa823675c1ece13fb76884622"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.000590 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "447463d2d30d4d8cb13031da4eff9a62"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "1125745479cdf537334501b2d6f26d40"
server_key_iv: "03c8c1eaa823675c1ece13fb76884622"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.004117 4473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:22.006255 4494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.007388 4473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:22.007489 4473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "447463d2d30d4d8cb13031da4eff9a62"
format_stamp: "Formatted at 2026-08-12 11:16:21 on dist-test-slave-gjrz"
server_key: "1125745479cdf537334501b2d6f26d40"
server_key_iv: "03c8c1eaa823675c1ece13fb76884622"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.007582 4473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:22.030869 4473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:22.033528 4473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:22.033697 4473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:22.034229 4473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:22.034997 4473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:22.035043 4473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.035089 4473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:22.035118 4473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.043448 4473 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:36087
I20260812 11:16:22.043481 4725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:36087 every 8 connection(s)
I20260812 11:16:22.044220 4473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:22.052651 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4473
I20260812 11:16:22.052749 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:22.052991 31742 external_mini_cluster.cc:1428] Setting key 3b0f5e7e53e7df1d196f2b98fcd8476a
I20260812 11:16:22.056356 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:22.045778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:51387 (local address 127.30.255.190:35057)
0812 11:16:22.045931 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:22.045935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:22.046683 (+ 748us) server_negotiation.cc:407] Connection header received
0812 11:16:22.047936 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:22.047938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:22.047984 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:22.048052 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:22.049075 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.049568 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.050601 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.050799 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.050861 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:22.053042 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:22.053045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:22.053047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:22.053076 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:22.054469 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:22.054922 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:22.054925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:22.054927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:22.055003 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:22.055356 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:22.055360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:22.055361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:22.055542 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:22.055628 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:22.056094 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 11:16:22.056205 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:16:22.057044 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:22.046130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:35057 (local address 127.30.255.130:51387)
0812 11:16:22.046544 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:22.046584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:22.047763 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:22.048224 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:22.048231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:22.048542 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:22.048968 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:22.048982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.049660 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.049662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:22.050505 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:22.050514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.050897 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.050900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:22.051028 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:22.051504 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:22.051519 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:22.052883 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:22.054572 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:22.054578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:22.054598 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:22.054830 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:22.055085 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:22.055088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:22.055090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:22.055236 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:22.055648 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:22.055654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:22.055864 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 11:16:22.056008 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 11:16:22.056289 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
I20260812 11:16:22.058110 4728 heartbeater.cc:347] Connected to a master server at 127.30.255.190:35057
I20260812 11:16:22.058315 4728 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:22.058749 4728 heartbeater.cc:511] Master 127.30.255.190:35057 requested a full tablet report, sending...
I20260812 11:16:22.059700 4091 ts_manager.cc:194] Registered new tserver with Master: 447463d2d30d4d8cb13031da4eff9a62 (127.30.255.130:36087)
I20260812 11:16:22.060230 4091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:51387
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:22.078217 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:22.184085 4739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:22.184301 4739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:22.184340 4739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:22.188131 4739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:22.188177 4739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:22.188241 4739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:22.193189 4739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:35057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:22.194041 4739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:22.194919 4739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:22.203634 4750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:22.203701 4748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:22.203826 4739 server_base.cc:1034] running on GCE node
W20260812 11:16:22.203683 4747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:22.204290 4739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:22.204885 4739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:22.206023 4739 hybrid_clock.cc:648] HybridClock initialized: now 1786533382205995 us; error 41 us; skew 500 ppm
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:22.208531 4739 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:22.209342 4739 webserver.cc:460] Webserver started at http://127.30.255.131:39957/ using document root <none> and password file <none>
I20260812 11:16:22.209867 4739 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:22.209932 4739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:22.210131 4739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:22.212087 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "a7d2393efb8946fa9ce4103a9fc82e2f"
format_stamp: "Formatted at 2026-08-12 11:16:22 on dist-test-slave-gjrz"
server_key: "45bf49af1aa550cadc721d3d016c2490"
server_key_iv: "d8b2f62671b3a61c39e5bab7a8eb612d"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.212620 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "a7d2393efb8946fa9ce4103a9fc82e2f"
format_stamp: "Formatted at 2026-08-12 11:16:22 on dist-test-slave-gjrz"
server_key: "45bf49af1aa550cadc721d3d016c2490"
server_key_iv: "d8b2f62671b3a61c39e5bab7a8eb612d"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.216009 4739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:16:22.218158 4767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.219209 4739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:22.219337 4739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "a7d2393efb8946fa9ce4103a9fc82e2f"
format_stamp: "Formatted at 2026-08-12 11:16:22 on dist-test-slave-gjrz"
server_key: "45bf49af1aa550cadc721d3d016c2490"
server_key_iv: "d8b2f62671b3a61c39e5bab7a8eb612d"
server_key_version: "encryptionkey@0"
I20260812 11:16:22.219430 4739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:22.234879 4739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:22.237231 4739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:22.237382 4739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:22.237869 4739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:22.238579 4739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:22.238626 4739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.238674 4739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:22.238704 4739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:22.246905 4739 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:34755
I20260812 11:16:22.246934 4995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:34755 every 8 connection(s)
I20260812 11:16:22.247665 4739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:22.253470 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4739
I20260812 11:16:22.253575 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:22.253829 31742 external_mini_cluster.cc:1428] Setting key 6f956385308f7ae0f65837172b460eba
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:22.259735 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:22.249230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:40355 (local address 127.30.255.190:35057)
0812 11:16:22.249358 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:22.249361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:22.250046 (+ 685us) server_negotiation.cc:407] Connection header received
0812 11:16:22.251305 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:22.251309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:22.251367 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:22.251437 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:22.252484 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.252999 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.254221 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.254499 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.254610 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:22.256915 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:22.256918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:22.256920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:22.256946 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:22.258194 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:22.258587 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:22.258590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:22.258591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:22.258661 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:22.258892 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:22.258893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:22.258894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:22.259055 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:22.259123 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:22.259535 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 11:16:22.259625 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:16:22.260386 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:22.249519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:35057 (local address 127.30.255.131:40355)
0812 11:16:22.249919 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:22.249955 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:22.251091 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:22.251605 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:22.251611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:22.251922 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:22.252347 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:22.252360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.253118 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.253121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:22.254094 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:22.254108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:22.254647 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:22.254651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:22.254815 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:22.255364 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:22.255384 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:22.256774 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:22.258285 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:22.258289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:22.258298 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:22.258518 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:22.258720 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:22.258722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:22.258723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:22.258833 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:22.259123 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:22.259127 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:22.259354 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 11:16:22.259498 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 11:16:22.259721 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":95,"threads_started":1}
I20260812 11:16:22.261261 4996 heartbeater.cc:347] Connected to a master server at 127.30.255.190:35057
I20260812 11:16:22.261437 4996 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:22.261821 4996 heartbeater.cc:511] Master 127.30.255.190:35057 requested a full tablet report, sending...
I20260812 11:16:22.262684 4091 ts_manager.cc:194] Registered new tserver with Master: a7d2393efb8946fa9ce4103a9fc82e2f (127.30.255.131:34755)
I20260812 11:16:22.263204 4091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:40355
I20260812 11:16:22.267104 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:16:22.280570 4163 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:22.279400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.30.255.190:35057)
0812 11:16:22.279511 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:22.279514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:22.279717 (+ 203us) server_negotiation.cc:407] Connection header received
0812 11:16:22.279813 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:22.279815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:22.279851 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:22.279912 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:22.280471 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55566: BlockingRecv error: recv got EOF from 127.0.0.1:55566 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:16:22.281623 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4213
I20260812 11:16:22.299588 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4473
I20260812 11:16:22.319557 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4739
I20260812 11:16:22.358609 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4026
2026-08-12T11:16:22Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2243 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[5018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[5018](info): set up 2 sockets
Aug 12 11:16:22 dist-test-slave-gjrz krb5kdc[5018](info): commencing operation
krb5kdc: starting...
W20260812 11:16:23.557442 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:23.668381 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38097
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:40797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:38097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:23.774958 5045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:23.775193 5045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:23.775249 5045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:23.779088 5045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:23.779137 5045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:23.779155 5045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:23.779174 5045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:23.779194 5045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:23.784113 5045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:38097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:23.784974 5045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:23.785846 5045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:23.793566 5058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:23.793566 5060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:23.793762 5057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:23.793804 5045 server_base.cc:1034] running on GCE node
I20260812 11:16:23.794322 5045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:23.795311 5045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:23.796471 5045 hybrid_clock.cc:648] HybridClock initialized: now 1786533383796456 us; error 24 us; skew 500 ppm
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533383, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:23.803851 5045 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:23.804687 5045 webserver.cc:460] Webserver started at http://127.30.255.190:34719/ using document root <none> and password file <none>
I20260812 11:16:23.805207 5045 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:23.805265 5045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:23.805455 5045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:23.807420 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "903897e1db8e4fa38f185e455182f2ba"
format_stamp: "Formatted at 2026-08-12 11:16:23 on dist-test-slave-gjrz"
server_key: "ab3de3a67f027de05552ea28d919329e"
server_key_iv: "54bc7bd80356321503e8feda2b0b03c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:23.807958 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "903897e1db8e4fa38f185e455182f2ba"
format_stamp: "Formatted at 2026-08-12 11:16:23 on dist-test-slave-gjrz"
server_key: "ab3de3a67f027de05552ea28d919329e"
server_key_iv: "54bc7bd80356321503e8feda2b0b03c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:23.811427 5045 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:16:23.813675 5074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:23.814762 5045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:23.814862 5045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "903897e1db8e4fa38f185e455182f2ba"
format_stamp: "Formatted at 2026-08-12 11:16:23 on dist-test-slave-gjrz"
server_key: "ab3de3a67f027de05552ea28d919329e"
server_key_iv: "54bc7bd80356321503e8feda2b0b03c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:23.814951 5045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:23.826725 5045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:23.829442 5045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:23.829591 5045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:23.835873 5045 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:38097
I20260812 11:16:23.836117 5162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:38097 every 8 connection(s)
I20260812 11:16:23.836704 5045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:23.839210 5164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:23.843364 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5045
I20260812 11:16:23.843482 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:23.843755 31742 external_mini_cluster.cc:1428] Setting key 8117c98c552857ca7f78c002f33318b4
I20260812 11:16:23.844736 5164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: Bootstrap starting.
I20260812 11:16:23.846894 5164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:23.847755 5164 log.cc:824] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:23.849709 5164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: No bootstrap required, opened a new log
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:23.853658 5164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } }
I20260812 11:16:23.853919 5164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:23.853991 5164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903897e1db8e4fa38f185e455182f2ba, State: Initialized, Role: FOLLOWER
I20260812 11:16:23.854378 5164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } }
I20260812 11:16:23.854503 5164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:23.854574 5164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:23.854669 5164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:23.855453 5164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } }
I20260812 11:16:23.855754 5164 leader_election.cc:302] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 903897e1db8e4fa38f185e455182f2ba; no voters:
I20260812 11:16:23.856012 5164 leader_election.cc:288] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:23.856101 5170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:23.856344 5170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [term 1 LEADER]: Becoming Leader. State: Replica: 903897e1db8e4fa38f185e455182f2ba, State: Running, Role: LEADER
I20260812 11:16:23.856668 5170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } }
I20260812 11:16:23.857267 5164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:23.857352 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:23.844951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.30.255.190:38097)
0812 11:16:23.845333 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:23.845352 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:23.845387 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:16:23.845975 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:23.845991 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:23.846288 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:23.846537 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:23.847533 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:23.848671 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:23.849778 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:23.850245 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:23.850364 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:23.852952 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:23.852957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:23.852969 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:23.852993 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:23.854593 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:23.855019 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:23.855025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:23.855030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:23.855140 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:23.855421 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:23.855425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:23.855427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:23.855848 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:23.856053 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:23.856286 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 11:16:23.856651 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260812 11:16:23.857769 5170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 903897e1db8e4fa38f185e455182f2ba. Latest consensus state: current_term: 1 leader_uuid: "903897e1db8e4fa38f185e455182f2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } } }
I20260812 11:16:23.857963 5170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:23.857808 5173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "903897e1db8e4fa38f185e455182f2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903897e1db8e4fa38f185e455182f2ba" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38097 } } }
I20260812 11:16:23.858274 5173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:23.860271 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 11:16:23.862124 5200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:23.862201 5200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:23.862268 5184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:23.862928 5184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:23.867589 5184 catalog_manager.cc:1347] Generated new cluster ID: 28f8b8ac6514493dbbaf74bbdd7d7198
I20260812 11:16:23.867650 5184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:23.881038 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:23.875957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.30.255.190:38097)
0812 11:16:23.876045 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:23.876049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:23.876292 (+ 243us) server_negotiation.cc:407] Connection header received
0812 11:16:23.876386 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:23.876389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:23.876434 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:23.876508 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:23.877125 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:23.877864 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:23.878966 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:23.879346 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:23.879426 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:23.880197 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:23.880201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:23.880203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:23.880654 (+ 451us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:23.880683 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:23.880699 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:23.880773 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:23.880867 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 11:16:23.880917 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:16:23.885429 5184 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:23.886271 5184 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:23.889904 5184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 903897e1db8e4fa38f185e455182f2ba: Generated new TSK 0
I20260812 11:16:23.890561 5184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:16:23.894918 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5045
2026-08-12T11:16:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1545 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5230](info): set up 2 sockets
Aug 12 11:16:23 dist-test-slave-gjrz krb5kdc[5230](info): commencing operation
krb5kdc: starting...
W20260812 11:16:25.101332 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:25.214090 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41723
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:25.318961 5250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:25.319180 5250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:25.319217 5250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:25.323053 5250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:25.323105 5250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:25.323123 5250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:25.323138 5250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:25.323153 5250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:25.328126 5250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:25.328977 5250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:25.329869 5250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:25.337623 5268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.337622 5273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.337688 5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:25.337980 5250 server_base.cc:1034] running on GCE node
I20260812 11:16:25.338471 5250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:25.339457 5250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:25.340616 5250 hybrid_clock.cc:648] HybridClock initialized: now 1786533385340597 us; error 28 us; skew 500 ppm
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:25.343091 5250 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:25.343938 5250 webserver.cc:460] Webserver started at http://127.30.255.190:36849/ using document root <none> and password file <none>
I20260812 11:16:25.344451 5250 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:25.344516 5250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:25.344731 5250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:25.346657 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "50a0f4ac453241d086dd0767f79133ef"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "bdf2874aebcccad2d9a2efce47c907eb"
server_key_iv: "aec52a908cf2c7095cf5e4218364e9a6"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.347173 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "50a0f4ac453241d086dd0767f79133ef"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "bdf2874aebcccad2d9a2efce47c907eb"
server_key_iv: "aec52a908cf2c7095cf5e4218364e9a6"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.350642 5250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:25.352854 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.353968 5250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:25.354089 5250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "50a0f4ac453241d086dd0767f79133ef"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "bdf2874aebcccad2d9a2efce47c907eb"
server_key_iv: "aec52a908cf2c7095cf5e4218364e9a6"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.354176 5250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:25.371552 5250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:25.373941 5250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:25.374099 5250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:25.380626 5250 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41723
I20260812 11:16:25.380867 5379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41723 every 8 connection(s)
I20260812 11:16:25.381461 5250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:25.384014 5381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:25.388865 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5250
I20260812 11:16:25.388981 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:25.389174 5381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: Bootstrap starting.
I20260812 11:16:25.389227 31742 external_mini_cluster.cc:1428] Setting key 97d8ad60c1e6e0f8f388c5e46de32dc1
I20260812 11:16:25.391433 5381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:25.392136 5381 log.cc:824] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:25.393893 5381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: No bootstrap required, opened a new log
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:25.397907 5381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } }
I20260812 11:16:25.398109 5381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:25.398180 5381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a0f4ac453241d086dd0767f79133ef, State: Initialized, Role: FOLLOWER
I20260812 11:16:25.398635 5381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } }
I20260812 11:16:25.398806 5381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:25.398875 5381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:25.398978 5381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:25.399978 5381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } }
I20260812 11:16:25.400308 5381 leader_election.cc:302] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50a0f4ac453241d086dd0767f79133ef; no voters:
I20260812 11:16:25.400602 5381 leader_election.cc:288] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:25.400674 5391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:25.400872 5391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [term 1 LEADER]: Becoming Leader. State: Replica: 50a0f4ac453241d086dd0767f79133ef, State: Running, Role: LEADER
I20260812 11:16:25.401175 5391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } }
I20260812 11:16:25.401789 5381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:25.402277 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50a0f4ac453241d086dd0767f79133ef. Latest consensus state: current_term: 1 leader_uuid: "50a0f4ac453241d086dd0767f79133ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } } }
I20260812 11:16:25.402391 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:25.403323 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:25.390442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.30.255.190:41723)
0812 11:16:25.390819 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:25.390833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:25.390866 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:16:25.391485 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:25.391503 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:25.391827 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:25.392092 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:25.392908 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.393860 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.394991 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.395505 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.395624 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:25.398202 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:25.398208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:25.398224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:25.398255 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:25.400081 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.400480 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.400486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.400492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.400632 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.401114 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.401117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.401119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.401574 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:25.401756 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:25.402073 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 11:16:25.402356 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260812 11:16:25.404599 5392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50a0f4ac453241d086dd0767f79133ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a0f4ac453241d086dd0767f79133ef" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41723 } } }
I20260812 11:16:25.404701 5392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef [sys.catalog]: This master's current role is: LEADER
W20260812 11:16:25.406461 5421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:25.406522 5421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:25.406622 5403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:25.407338 5403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:25.412568 5403 catalog_manager.cc:1347] Generated new cluster ID: 28d61600f774443ca24e6a4cfd9cab4a
I20260812 11:16:25.412626 5403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:25.424782 5403 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:25.425559 5403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:25.432134 5403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50a0f4ac453241d086dd0767f79133ef: Generated new TSK 0
I20260812 11:16:25.432668 5403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:25.464462 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:25.574352 5432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:25.574564 5432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:25.574606 5432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:25.578460 5432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:25.578507 5432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:25.578573 5432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:25.583689 5432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:25.584579 5432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:25.585496 5432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:25.594178 5452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.594206 5448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:25.594273 5432 server_base.cc:1034] running on GCE node
W20260812 11:16:25.594206 5443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:25.594745 5432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:25.595330 5432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:25.596488 5432 hybrid_clock.cc:648] HybridClock initialized: now 1786533385596455 us; error 43 us; skew 500 ppm
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:25.598981 5432 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:25.599788 5432 webserver.cc:460] Webserver started at http://127.30.255.129:35469/ using document root <none> and password file <none>
I20260812 11:16:25.600279 5432 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:25.600337 5432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:25.600517 5432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:25.602432 5432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "91cf8567604444b2b5ce6f72e6f47bea"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "b99369d514dabb46f11be6d6970fc5b8"
server_key_iv: "9e184b86b96fdfb6e4a4af2d2d129c27"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.602962 5432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "91cf8567604444b2b5ce6f72e6f47bea"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "b99369d514dabb46f11be6d6970fc5b8"
server_key_iv: "9e184b86b96fdfb6e4a4af2d2d129c27"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.606447 5432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:25.608632 5464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.609717 5432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:25.609818 5432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "91cf8567604444b2b5ce6f72e6f47bea"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "b99369d514dabb46f11be6d6970fc5b8"
server_key_iv: "9e184b86b96fdfb6e4a4af2d2d129c27"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.609905 5432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:25.621480 5432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:25.623929 5432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:25.624082 5432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:25.624583 5432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:25.625341 5432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:25.625398 5432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.625443 5432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:25.625471 5432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.633443 5432 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:32807
I20260812 11:16:25.633469 5680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:32807 every 8 connection(s)
I20260812 11:16:25.634203 5432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:25.639869 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5432
I20260812 11:16:25.639988 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:25.640273 31742 external_mini_cluster.cc:1428] Setting key 93b943ff3ef0916cdb31ccfcbd25ef92
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:25.647161 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:25.635783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:33023 (local address 127.30.255.190:41723)
0812 11:16:25.635935 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:25.635940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:25.636681 (+ 741us) server_negotiation.cc:407] Connection header received
0812 11:16:25.637922 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:25.637925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:25.637970 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:25.638039 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:25.639047 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.639594 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.641035 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.641316 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.641407 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:25.643943 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:25.643946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:25.643948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:25.643970 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:25.645214 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.645755 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.645757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.645758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.645827 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.646180 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.646182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.646183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.646330 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:25.646397 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:25.646951 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 11:16:25.647043 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":57}
I20260812 11:16:25.648231 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:25.636131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41723 (local address 127.30.255.129:33023)
0812 11:16:25.636546 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:25.636585 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:25.637748 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:25.638189 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:25.638196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:25.638511 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:25.638938 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:25.638951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.639711 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.639715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:25.640928 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:25.640940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.641458 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.641463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:25.641627 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:25.642173 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:25.642191 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:25.643785 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:25.645329 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:25.645334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:25.645348 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:25.645674 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:25.645904 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:25.645908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:25.645910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:25.646103 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:25.646426 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:25.646433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:25.646730 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 11:16:25.646935 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 11:16:25.647247 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
I20260812 11:16:25.649354 5681 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41723
I20260812 11:16:25.649560 5681 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:25.649978 5681 heartbeater.cc:511] Master 127.30.255.190:41723 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:25.651412 5314 ts_manager.cc:194] Registered new tserver with Master: 91cf8567604444b2b5ce6f72e6f47bea (127.30.255.129:32807)
I20260812 11:16:25.652525 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:33023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:25.666044 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:25.773484 5686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:25.773697 5686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:25.773736 5686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:25.777590 5686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:25.777638 5686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:25.777690 5686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:25.782548 5686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:25.783396 5686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:25.784245 5686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:25.792831 5703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.792832 5699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.792900 5696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:25.793006 5686 server_base.cc:1034] running on GCE node
I20260812 11:16:25.793485 5686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:25.794044 5686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:25.795200 5686 hybrid_clock.cc:648] HybridClock initialized: now 1786533385795185 us; error 26 us; skew 500 ppm
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:25.797668 5686 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:25.798403 5686 webserver.cc:460] Webserver started at http://127.30.255.130:46537/ using document root <none> and password file <none>
I20260812 11:16:25.798884 5686 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:25.798947 5686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:25.799120 5686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:25.801024 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "705120d2aad94781824fb1800cfbeb8b"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "59a4cd9a6179c6d45b53f077594487f0"
server_key_iv: "6adbc1720884c17d3f913911ce6d9b89"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.801575 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "705120d2aad94781824fb1800cfbeb8b"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "59a4cd9a6179c6d45b53f077594487f0"
server_key_iv: "6adbc1720884c17d3f913911ce6d9b89"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.805027 5686 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:16:25.807148 5718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.808632 5686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:25.808749 5686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "705120d2aad94781824fb1800cfbeb8b"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "59a4cd9a6179c6d45b53f077594487f0"
server_key_iv: "6adbc1720884c17d3f913911ce6d9b89"
server_key_version: "encryptionkey@0"
I20260812 11:16:25.808859 5686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:25.825503 5686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:25.828208 5686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:25.828362 5686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:25.828862 5686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:25.829604 5686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:25.829650 5686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.829699 5686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:25.829725 5686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:25.837361 5686 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:39889
I20260812 11:16:25.837412 5950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:39889 every 8 connection(s)
I20260812 11:16:25.838125 5686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:25.841514 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5686
I20260812 11:16:25.841635 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:25.841941 31742 external_mini_cluster.cc:1428] Setting key 738ee7b04b53ecfe7179da5d736eadda
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:25.851058 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:25.839661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:36471 (local address 127.30.255.190:41723)
0812 11:16:25.839777 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:25.839781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:25.840493 (+ 712us) server_negotiation.cc:407] Connection header received
0812 11:16:25.841781 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:25.841785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:25.841844 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:25.841951 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:25.843063 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.843674 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.845129 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.845338 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.845424 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:25.847723 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:25.847726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:25.847728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:25.847751 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:25.849211 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.849669 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.849673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.849675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.849769 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:25.850048 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:25.850051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:25.850053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:25.850267 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:25.850371 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:25.850778 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 11:16:25.850895 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:25.851686 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:25.839956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41723 (local address 127.30.255.130:36471)
0812 11:16:25.840361 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:25.840398 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:25.841565 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:25.842102 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:25.842108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:25.842416 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:25.842907 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:25.842921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.843805 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.843808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:25.845017 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:25.845028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:25.845477 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:25.845482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:25.845647 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:25.846104 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:25.846119 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:25.847571 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:25.849322 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:25.849326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:25.849336 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:25.849576 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:25.849849 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:25.849851 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:25.849853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:25.849966 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:25.850357 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:25.850361 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:25.850559 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 11:16:25.850699 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 11:16:25.850914 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:25.852689 5952 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41723
I20260812 11:16:25.852870 5952 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:25.853315 5952 heartbeater.cc:511] Master 127.30.255.190:41723 requested a full tablet report, sending...
I20260812 11:16:25.854213 5314 ts_manager.cc:194] Registered new tserver with Master: 705120d2aad94781824fb1800cfbeb8b (127.30.255.130:39889)
I20260812 11:16:25.854753 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:36471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:25.869562 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:25.975363 5969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:25.975580 5969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:25.975618 5969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:25.979684 5969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:25.979732 5969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:25.979785 5969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:25.984671 5969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:25.985478 5969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:25.986320 5969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:25.995191 5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.995223 5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:25.995221 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:25.995514 5969 server_base.cc:1034] running on GCE node
I20260812 11:16:25.995886 5969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:25.996405 5969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:25.997552 5969 hybrid_clock.cc:648] HybridClock initialized: now 1786533385997536 us; error 25 us; skew 500 ppm
Aug 12 11:16:25 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:26.000049 5969 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:26.000831 5969 webserver.cc:460] Webserver started at http://127.30.255.131:38043/ using document root <none> and password file <none>
I20260812 11:16:26.001320 5969 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:26.001384 5969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:26.001574 5969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:26.003578 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "ec3cfbcb7d99415cb703bb73f57d1d1c"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "d15c2843f2fabb09565f668dbe21dcf0"
server_key_iv: "3d0e700fc831f32dbc1dcb8768de95fa"
server_key_version: "encryptionkey@0"
I20260812 11:16:26.004099 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "ec3cfbcb7d99415cb703bb73f57d1d1c"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "d15c2843f2fabb09565f668dbe21dcf0"
server_key_iv: "3d0e700fc831f32dbc1dcb8768de95fa"
server_key_version: "encryptionkey@0"
I20260812 11:16:26.007473 5969 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:16:26.009618 5994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:26.010694 5969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:26.010807 5969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "ec3cfbcb7d99415cb703bb73f57d1d1c"
format_stamp: "Formatted at 2026-08-12 11:16:25 on dist-test-slave-gjrz"
server_key: "d15c2843f2fabb09565f668dbe21dcf0"
server_key_iv: "3d0e700fc831f32dbc1dcb8768de95fa"
server_key_version: "encryptionkey@0"
I20260812 11:16:26.010890 5969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:26.027135 5969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:26.029923 5969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:26.030086 5969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:26.030606 5969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:26.031365 5969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:26.031411 5969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:26.031462 5969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:26.031490 5969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:26.039373 5969 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:38057
I20260812 11:16:26.039404 6212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:38057 every 8 connection(s)
I20260812 11:16:26.040148 5969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:26.044780 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5969
I20260812 11:16:26.044891 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:26.045186 31742 external_mini_cluster.cc:1428] Setting key fb760269d8d091237c754ca7940bf6da
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[5230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:26.052487 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:26.041741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:52239 (local address 127.30.255.190:41723)
0812 11:16:26.041864 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:26.041868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:26.042612 (+ 744us) server_negotiation.cc:407] Connection header received
0812 11:16:26.043901 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:26.043903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:26.043949 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:26.044010 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:26.045211 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.045811 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.047285 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.047500 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.047578 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:26.049626 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:26.049630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:26.049632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:26.049656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:26.050901 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:26.051338 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:26.051341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:26.051342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:26.051413 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:26.051668 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:26.051669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:26.051670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:26.051820 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:26.051887 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:26.052290 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 11:16:26.052376 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:16:26.053182 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:26.042056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41723 (local address 127.30.255.131:52239)
0812 11:16:26.042463 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:26.042502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:26.043711 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:26.044150 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:26.044157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:26.044460 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:26.045070 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:26.045090 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.045929 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.045933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:26.047160 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:26.047170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.047613 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.047616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:26.047749 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:26.048170 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:26.048185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:26.049487 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:26.050982 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:26.050986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:26.050995 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:26.051225 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:26.051479 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:26.051481 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:26.051483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:26.051603 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:26.051887 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:26.051891 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:26.052118 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 11:16:26.052259 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 11:16:26.052474 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":109,"threads_started":1}
I20260812 11:16:26.054148 6213 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41723
I20260812 11:16:26.054327 6213 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:26.054725 6213 heartbeater.cc:511] Master 127.30.255.190:41723 requested a full tablet report, sending...
I20260812 11:16:26.055620 5314 ts_manager.cc:194] Registered new tserver with Master: ec3cfbcb7d99415cb703bb73f57d1d1c (127.30.255.131:38057)
I20260812 11:16:26.056114 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:52239
I20260812 11:16:26.058594 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[5230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533386, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[5230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533386, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:26.082356 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:26.075392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.30.255.190:41723)
0812 11:16:26.075493 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:26.075496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:26.075660 (+ 164us) server_negotiation.cc:407] Connection header received
0812 11:16:26.075757 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:26.075759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:26.075796 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:26.075861 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:26.076457 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.076934 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.077662 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:26.077858 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:26.077922 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:26.079945 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:26.079948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:26.079950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:26.079972 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:26.081168 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:26.081493 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:26.081495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:26.081496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:26.081560 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:26.081733 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:26.081735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:26.081736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:26.081926 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:26.082025 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:26.082167 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 11:16:26.082250 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 11:16:26.082693 5314 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53400
I20260812 11:16:26.083424 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5432
I20260812 11:16:26.110303 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5686
I20260812 11:16:26.123234 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5969
I20260812 11:16:26.134356 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5250
2026-08-12T11:16:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2233 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[6239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[6239](info): set up 2 sockets
Aug 12 11:16:26 dist-test-slave-gjrz krb5kdc[6239](info): commencing operation
krb5kdc: starting...
W20260812 11:16:27.357852 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:27 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:27.470137 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37427
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:37427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:27.578788 6276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:27.579021 6276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:27.579075 6276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:27.582942 6276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:27.582993 6276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:27.583019 6276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:27.583039 6276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:27.583063 6276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:27.588012 6276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:37427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:27.588901 6276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:27.589819 6276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:27.597586 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:27.597609 6290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:27.597636 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:27.597903 6276 server_base.cc:1034] running on GCE node
I20260812 11:16:27.598413 6276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:27.599442 6276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:27.600593 6276 hybrid_clock.cc:648] HybridClock initialized: now 1786533387600576 us; error 29 us; skew 500 ppm
Aug 12 11:16:27 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533387, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:27.603120 6276 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:27.603943 6276 webserver.cc:460] Webserver started at http://127.30.255.190:45169/ using document root <none> and password file <none>
I20260812 11:16:27.604420 6276 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:27.604476 6276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:27.604650 6276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:27.606489 6276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "35a885465c6640d09f449c7f7beee1ae"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "c7512fa02786d94cf84d725e0def6598"
server_key_iv: "1531e8bda8b8de4fd258a79048d6d6fe"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.607077 6276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "35a885465c6640d09f449c7f7beee1ae"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "c7512fa02786d94cf84d725e0def6598"
server_key_iv: "1531e8bda8b8de4fd258a79048d6d6fe"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.610514 6276 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:27.612726 6306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:27.613821 6276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:27.613922 6276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "35a885465c6640d09f449c7f7beee1ae"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "c7512fa02786d94cf84d725e0def6598"
server_key_iv: "1531e8bda8b8de4fd258a79048d6d6fe"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.614014 6276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:27.624574 6276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:27.627100 6276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:27.627255 6276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:27.633659 6276 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:37427
I20260812 11:16:27.633672 6419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:37427 every 8 connection(s)
I20260812 11:16:27.634502 6276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:27.635236 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6276
I20260812 11:16:27.635383 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:27.635676 31742 external_mini_cluster.cc:1428] Setting key ed7b058a0dacf366d267587427c54fb2
I20260812 11:16:27.637642 6420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:27.642974 6420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae: Bootstrap starting.
Aug 12 11:16:27 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:27.645568 6420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:27.646286 6420 log.cc:824] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:27.648372 6420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae: No bootstrap required, opened a new log
I20260812 11:16:27.648805 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:27.636995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.30.255.190:37427)
0812 11:16:27.637409 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:27.637424 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:27.637457 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:16:27.638097 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:27.638114 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:27.638419 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:27.638720 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:27.639522 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.640473 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.641300 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.641756 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.641923 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:27.644524 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:27.644529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:27.644542 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:27.644570 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:27.646087 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:27.646559 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:27.646564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:27.646568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:27.646691 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:27.646958 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:27.646961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:27.646962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:27.647416 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:27.647559 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:27.647832 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 11:16:27.648098 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":148,"threads_started":1}
I20260812 11:16:27.652014 6420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } }
I20260812 11:16:27.652216 6420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:27.652276 6420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a885465c6640d09f449c7f7beee1ae, State: Initialized, Role: FOLLOWER
I20260812 11:16:27.652657 6420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } }
I20260812 11:16:27.652776 6420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:27.652837 6420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:27.652926 6420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:27.653707 6420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } }
I20260812 11:16:27.654016 6420 leader_election.cc:302] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35a885465c6640d09f449c7f7beee1ae; no voters:
I20260812 11:16:27.654287 6420 leader_election.cc:288] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:27.654386 6425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:27.654605 6425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [term 1 LEADER]: Becoming Leader. State: Replica: 35a885465c6640d09f449c7f7beee1ae, State: Running, Role: LEADER
I20260812 11:16:27.654930 6425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } }
I20260812 11:16:27.655476 6420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:27.656577 6430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35a885465c6640d09f449c7f7beee1ae. Latest consensus state: current_term: 1 leader_uuid: "35a885465c6640d09f449c7f7beee1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } } }
I20260812 11:16:27.656629 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35a885465c6640d09f449c7f7beee1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a885465c6640d09f449c7f7beee1ae" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37427 } } }
I20260812 11:16:27.656713 6430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:27.656725 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:27.657042 6444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:27.659549 6444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:27.664832 6444 catalog_manager.cc:1347] Generated new cluster ID: 9e7f94e850244274ae1747f860557068
I20260812 11:16:27.664883 6444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:27.676918 6444 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:27.677773 6444 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:27.684465 6444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35a885465c6640d09f449c7f7beee1ae: Generated new TSK 0
I20260812 11:16:27.685019 6444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:27.722489 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:27.847543 6462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:27.847754 6462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:27.847791 6462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:27.851601 6462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:27.851644 6462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:27.851696 6462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:27.856572 6462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:27.857439 6462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:27.858354 6462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:27.867174 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:27.867105 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:27.867125 6474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:27.867148 6462 server_base.cc:1034] running on GCE node
I20260812 11:16:27.867758 6462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:27.868296 6462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:27.869481 6462 hybrid_clock.cc:648] HybridClock initialized: now 1786533387869451 us; error 39 us; skew 500 ppm
Aug 12 11:16:27 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533387, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:27.872094 6462 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:27.872834 6462 webserver.cc:460] Webserver started at http://127.30.255.129:33793/ using document root <none> and password file <none>
I20260812 11:16:27.873329 6462 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:27.873387 6462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:27.873560 6462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:27.875432 6462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "e3738a3ee254471d9546aae2a724fb82"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "27dac160f64797d3b5d300be1740f22d"
server_key_iv: "410097e4a9ddf887bbdf8ac6658d9692"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.875934 6462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "e3738a3ee254471d9546aae2a724fb82"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "27dac160f64797d3b5d300be1740f22d"
server_key_iv: "410097e4a9ddf887bbdf8ac6658d9692"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.879432 6462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:27.881619 6489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:27.882674 6462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:27.882776 6462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "e3738a3ee254471d9546aae2a724fb82"
format_stamp: "Formatted at 2026-08-12 11:16:27 on dist-test-slave-gjrz"
server_key: "27dac160f64797d3b5d300be1740f22d"
server_key_iv: "410097e4a9ddf887bbdf8ac6658d9692"
server_key_version: "encryptionkey@0"
I20260812 11:16:27.882860 6462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:27.925674 6462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:27.928121 6462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:27.928287 6462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:27.928809 6462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:27.929543 6462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:27.929591 6462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:27.929641 6462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:27.929666 6462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:27.937631 6462 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:37065
I20260812 11:16:27.937671 6732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:37065 every 8 connection(s)
I20260812 11:16:27.938419 6462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:27.938692 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6462
I20260812 11:16:27.938799 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:27.939131 31742 external_mini_cluster.cc:1428] Setting key 0df0eb4adc6dbdf99ff92a943d6ad807
Aug 12 11:16:27 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533387, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:27.951318 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:27.940249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:58851 (local address 127.30.255.190:37427)
0812 11:16:27.940389 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:27.940394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:27.941232 (+ 838us) server_negotiation.cc:407] Connection header received
0812 11:16:27.942578 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:27.942581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:27.942635 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:27.942710 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:27.943856 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.944390 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.945501 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.945744 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.945835 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:27.948092 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:27.948095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:27.948097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:27.948122 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:27.949397 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:27.949863 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:27.949867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:27.949868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:27.949944 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:27.950252 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:27.950255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:27.950257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:27.950427 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:27.950510 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:27.950997 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 11:16:27.951133 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:16:27.952010 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:27.940631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37427 (local address 127.30.255.129:58851)
0812 11:16:27.941077 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:27.941126 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:27.942379 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:27.942884 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:27.942891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:27.943218 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:27.943711 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:27.943726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.944512 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.944514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:27.945381 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:27.945391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:27.945882 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:27.945886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:27.946040 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:27.946538 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:27.946565 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:27.947928 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:27.949507 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:27.949512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:27.949526 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:27.949768 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:27.950028 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:27.950031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:27.950033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:27.950163 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:27.950527 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:27.950533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:27.950760 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 11:16:27.950941 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 11:16:27.951239 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
I20260812 11:16:27.953151 6734 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37427
I20260812 11:16:27.953370 6734 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:27.953832 6734 heartbeater.cc:511] Master 127.30.255.190:37427 requested a full tablet report, sending...
I20260812 11:16:27.955454 6342 ts_manager.cc:194] Registered new tserver with Master: e3738a3ee254471d9546aae2a724fb82 (127.30.255.129:37065)
I20260812 11:16:27.956995 6342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:58851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:27.965660 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:28.072669 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:28.072875 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:28.072912 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:28.076777 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:28.076824 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:28.076876 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:28.081719 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:28.082559 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:28.083452 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:28.092320 6750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:28.092329 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:28.092329 6757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:28.092360 6743 server_base.cc:1034] running on GCE node
I20260812 11:16:28.092871 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:28.093380 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:28.094542 6743 hybrid_clock.cc:648] HybridClock initialized: now 1786533388094522 us; error 30 us; skew 500 ppm
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:28.097033 6743 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:28.097783 6743 webserver.cc:460] Webserver started at http://127.30.255.130:42891/ using document root <none> and password file <none>
I20260812 11:16:28.098266 6743 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:28.098330 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:28.098505 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:28.100399 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "f192e939ee3c4b31b421e81197e09c9b"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "2ad04d1217d79b86e25aca492d3820f7"
server_key_iv: "00e7387bb494d7b4b4853ce892906a60"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.100904 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "f192e939ee3c4b31b421e81197e09c9b"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "2ad04d1217d79b86e25aca492d3820f7"
server_key_iv: "00e7387bb494d7b4b4853ce892906a60"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.104355 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:28.106417 6773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.107514 6743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:28.107630 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "f192e939ee3c4b31b421e81197e09c9b"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "2ad04d1217d79b86e25aca492d3820f7"
server_key_iv: "00e7387bb494d7b4b4853ce892906a60"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.107714 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:28.120357 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:28.123086 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:28.123243 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:28.123808 6743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:28.124513 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:28.124567 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.124615 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:28.124641 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.132776 6743 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:34769
I20260812 11:16:28.132817 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:34769 every 8 connection(s)
I20260812 11:16:28.133533 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:28.140939 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6743
I20260812 11:16:28.141050 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:28.141302 31742 external_mini_cluster.cc:1428] Setting key 00fa67383dfdb1acc870e06307120add
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:28.146209 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.135100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:34707 (local address 127.30.255.190:37427)
0812 11:16:28.135218 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:28.135222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:28.135993 (+ 771us) server_negotiation.cc:407] Connection header received
0812 11:16:28.137464 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:28.137468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:28.137542 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:28.137614 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:28.138708 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.139226 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.140350 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.140582 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.140673 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.143009 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:28.143013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:28.143014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:28.143038 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:28.144294 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.144776 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.144779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.144780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.144858 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.145174 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.145177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.145179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.145364 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:28.145449 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:28.145920 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 11:16:28.146043 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:16:28.146873 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.135442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37427 (local address 127.30.255.130:34707)
0812 11:16:28.135856 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:28.135893 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:28.137242 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:28.137797 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:28.137804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:28.138127 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:28.138566 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:28.138580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.139365 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.139369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:28.140228 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:28.140238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.140721 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.140725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:28.140876 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.141381 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:28.141397 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:28.142847 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:28.144407 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:28.144413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:28.144426 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:28.144678 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:28.144944 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:28.144947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:28.144949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:28.145079 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:28.145468 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:28.145474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:28.145696 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 11:16:28.145857 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:16:28.146147 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
I20260812 11:16:28.147926 6990 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37427
I20260812 11:16:28.148171 6990 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:28.148718 6990 heartbeater.cc:511] Master 127.30.255.190:37427 requested a full tablet report, sending...
I20260812 11:16:28.149816 6342 ts_manager.cc:194] Registered new tserver with Master: f192e939ee3c4b31b421e81197e09c9b (127.30.255.130:34769)
I20260812 11:16:28.150375 6342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:34707
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:28.167397 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:28.273592 6997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:28.273815 6997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:28.273871 6997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:28.277748 6997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:28.277802 6997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:28.277878 6997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:28.282893 6997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:28.283838 6997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:28.284874 6997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:28.293867 7015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:28.293900 7014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:28.293879 7018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:28.294127 6997 server_base.cc:1034] running on GCE node
I20260812 11:16:28.294481 6997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:28.295033 6997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:28.296183 6997 hybrid_clock.cc:648] HybridClock initialized: now 1786533388296162 us; error 31 us; skew 500 ppm
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:28.298695 6997 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:28.299475 6997 webserver.cc:460] Webserver started at http://127.30.255.131:42179/ using document root <none> and password file <none>
I20260812 11:16:28.299954 6997 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:28.300009 6997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:28.300161 6997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:28.302000 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "661fba92b93749fb8b02a81a0af89287"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "d7f3b41a5b90980c331b1811064e4957"
server_key_iv: "130c5a05dd710281f83d731f0282ff64"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.302485 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "661fba92b93749fb8b02a81a0af89287"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "d7f3b41a5b90980c331b1811064e4957"
server_key_iv: "130c5a05dd710281f83d731f0282ff64"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.306002 6997 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 11:16:28.308192 7029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.309255 6997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 11:16:28.309347 6997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "661fba92b93749fb8b02a81a0af89287"
format_stamp: "Formatted at 2026-08-12 11:16:28 on dist-test-slave-gjrz"
server_key: "d7f3b41a5b90980c331b1811064e4957"
server_key_iv: "130c5a05dd710281f83d731f0282ff64"
server_key_version: "encryptionkey@0"
I20260812 11:16:28.309424 6997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:28.324112 6997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:28.326476 6997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:28.326608 6997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:28.327075 6997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:28.327834 6997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:28.327881 6997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.327916 6997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:28.327935 6997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:28.335928 6997 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:34975
I20260812 11:16:28.335955 7255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:34975 every 8 connection(s)
I20260812 11:16:28.336699 6997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:28.342237 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6997
I20260812 11:16:28.342363 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:28.342617 31742 external_mini_cluster.cc:1428] Setting key fdd99e3071bab2261931323b2c64637d
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:28.349050 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.338296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:40691 (local address 127.30.255.190:37427)
0812 11:16:28.338406 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:28.338410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:28.339120 (+ 710us) server_negotiation.cc:407] Connection header received
0812 11:16:28.340477 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:28.340480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:28.340542 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:28.340613 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:28.341700 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.342237 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.343595 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.343841 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.343936 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.346320 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:28.346324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:28.346326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:28.346351 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:28.347488 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.347905 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.347907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.347908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.347975 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.348219 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.348220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.348221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.348376 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:28.348443 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:28.348837 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 11:16:28.348935 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:16:28.349714 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.338567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37427 (local address 127.30.255.131:40691)
0812 11:16:28.338979 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:28.339018 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:28.340273 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:28.340792 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:28.340799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:28.341115 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:28.341554 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:28.341568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.342360 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.342364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:28.343486 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:28.343498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.344004 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.344009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:28.344210 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.344795 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:28.344812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:28.346163 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:28.347578 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:28.347583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:28.347594 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:28.347829 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:28.348032 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:28.348034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:28.348035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:28.348156 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:28.348443 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:28.348447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:28.348647 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 11:16:28.348789 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 11:16:28.349006 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":108,"threads_started":1}
I20260812 11:16:28.350687 7256 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37427
I20260812 11:16:28.350890 7256 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:28.351351 7256 heartbeater.cc:511] Master 127.30.255.190:37427 requested a full tablet report, sending...
I20260812 11:16:28.352319 6342 ts_manager.cc:194] Registered new tserver with Master: 661fba92b93749fb8b02a81a0af89287 (127.30.255.131:34975)
I20260812 11:16:28.352846 6342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:40691
I20260812 11:16:28.356196 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:28.381117 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.372702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.30.255.129:37065)
0812 11:16:28.373010 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:28.373019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:28.373041 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:16:28.373087 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:28.373091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:28.373248 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:28.373358 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:28.373945 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.374697 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.375506 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.375859 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.375941 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.378029 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:28.378038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:28.378049 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:28.378072 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:28.379427 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.379830 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.379836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.379837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.379918 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.380175 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.380179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.380180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.380411 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:28.380616 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:28.380802 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 11:16:28.380950 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":104,"threads_started":1}
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[6239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:28.390425 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:28.383540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.30.255.190:37427)
0812 11:16:28.383670 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:28.383674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:28.383738 (+ 64us) server_negotiation.cc:407] Connection header received
0812 11:16:28.383859 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:28.383862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:28.383904 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:28.383963 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:28.384558 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.385039 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.385779 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:28.386041 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:28.386136 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:28.388166 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:28.388169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:28.388171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:28.388194 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:28.389230 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.389570 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.389572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.389573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.389635 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:28.389810 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:28.389811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:28.389812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:28.389991 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:28.390097 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:28.390237 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 11:16:28.390321 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 11:16:28.390760 6342 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49364
I20260812 11:16:28.391256 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6462
I20260812 11:16:28.415187 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6743
I20260812 11:16:28.426374 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6997
I20260812 11:16:28.441016 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6276
2026-08-12T11:16:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2302 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[7277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[7277](info): set up 2 sockets
Aug 12 11:16:28 dist-test-slave-gjrz krb5kdc[7277](info): commencing operation
krb5kdc: starting...
W20260812 11:16:29.639599 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:29 dist-test-slave-gjrz krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:29.757962 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:40875
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:40875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:29.866112 7300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:29.866343 7300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:29.866381 7300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:29.870222 7300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:29.870272 7300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:29.870289 7300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:29.870303 7300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:29.870317 7300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:29.875229 7300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:40875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:29.876111 7300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:29.877012 7300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:29.884716 7312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:29.884727 7317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:29.884946 7314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:29.884969 7300 server_base.cc:1034] running on GCE node
I20260812 11:16:29.885562 7300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:29.886564 7300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:29.887717 7300 hybrid_clock.cc:648] HybridClock initialized: now 1786533389887699 us; error 44 us; skew 500 ppm
Aug 12 11:16:29 dist-test-slave-gjrz krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533389, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:29.896492 7300 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:29.897166 7300 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:29.897233 7300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:29.897418 7300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:29.899406 7300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "870a980c28534978a36c36cb256ff0a6"
format_stamp: "Formatted at 2026-08-12 11:16:29 on dist-test-slave-gjrz"
server_key: "3e5a1bc71d395c80a814902b32424a84"
server_key_iv: "f5d02c4c1a08e23740da140789adba52"
server_key_version: "encryptionkey@0"
I20260812 11:16:29.899950 7300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "870a980c28534978a36c36cb256ff0a6"
format_stamp: "Formatted at 2026-08-12 11:16:29 on dist-test-slave-gjrz"
server_key: "3e5a1bc71d395c80a814902b32424a84"
server_key_iv: "f5d02c4c1a08e23740da140789adba52"
server_key_version: "encryptionkey@0"
I20260812 11:16:29.903417 7300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:16:29.905635 7328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:29.906759 7300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:29.906872 7300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "870a980c28534978a36c36cb256ff0a6"
format_stamp: "Formatted at 2026-08-12 11:16:29 on dist-test-slave-gjrz"
server_key: "3e5a1bc71d395c80a814902b32424a84"
server_key_iv: "f5d02c4c1a08e23740da140789adba52"
server_key_version: "encryptionkey@0"
I20260812 11:16:29.906960 7300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:29.922706 7300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:29.925215 7300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:29.925374 7300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:29.931814 7300 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:40875
I20260812 11:16:29.932073 7429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:40875 every 8 connection(s)
I20260812 11:16:29.932415 7300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:29.933419 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7300
I20260812 11:16:29.933555 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:29.933835 31742 external_mini_cluster.cc:1428] Setting key 147031ed371376aa823eba01186860ae
I20260812 11:16:29.935582 7431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:29.941634 7431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: Bootstrap starting.
Aug 12 11:16:29 dist-test-slave-gjrz krb5kdc[7277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:29.943886 7431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:29.944561 7431 log.cc:824] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:29.946266 7431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: No bootstrap required, opened a new log
I20260812 11:16:29.946939 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:29.935092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.30.255.190:40875)
0812 11:16:29.935498 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:29.935514 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:29.935548 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:16:29.936185 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:29.936203 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:29.936505 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:29.936772 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:29.937606 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:29.938557 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:29.939717 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:29.940171 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:29.940283 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:29.942796 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:29.942801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:29.942812 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:29.942838 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:29.944392 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:29.944804 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:29.944810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:29.944814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:29.944922 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:29.945182 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:29.945185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:29.945187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:29.945543 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:29.945716 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:29.946002 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 11:16:29.946267 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":150,"threads_started":1}
I20260812 11:16:29.949915 7431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } }
I20260812 11:16:29.950129 7431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:29.950187 7431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870a980c28534978a36c36cb256ff0a6, State: Initialized, Role: FOLLOWER
I20260812 11:16:29.950558 7431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } }
I20260812 11:16:29.950683 7431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:29.950745 7431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:29.950842 7431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:29.951712 7431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } }
I20260812 11:16:29.952024 7431 leader_election.cc:302] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 870a980c28534978a36c36cb256ff0a6; no voters:
I20260812 11:16:29.952299 7431 leader_election.cc:288] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:29.952404 7442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:29.952615 7442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 870a980c28534978a36c36cb256ff0a6, State: Running, Role: LEADER
I20260812 11:16:29.952991 7442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } }
I20260812 11:16:29.953447 7431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:29.954568 7443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "870a980c28534978a36c36cb256ff0a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } } }
I20260812 11:16:29.954579 7445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 870a980c28534978a36c36cb256ff0a6. Latest consensus state: current_term: 1 leader_uuid: "870a980c28534978a36c36cb256ff0a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870a980c28534978a36c36cb256ff0a6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40875 } } }
I20260812 11:16:29.954735 7443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:29.954771 7445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:29.955147 7452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:29.958133 7452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:29.962877 7452 catalog_manager.cc:1347] Generated new cluster ID: 6debdedeb8a14f9695bc3755626bb4e3
I20260812 11:16:29.962940 7452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:30.011967 7452 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:30.012876 7452 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:30.020390 7452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: Generated new TSK 0
I20260812 11:16:30.021004 7452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:30.053750 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:30.162096 7488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:30.162312 7488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:30.162351 7488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:30.166285 7488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:30.166333 7488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:30.166399 7488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:30.171427 7488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:30.172293 7488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:30.173204 7488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:30.181841 7497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.181813 7495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.181818 7501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:30.182071 7488 server_base.cc:1034] running on GCE node
I20260812 11:16:30.182520 7488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:30.183051 7488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:30.184211 7488 hybrid_clock.cc:648] HybridClock initialized: now 1786533390184186 us; error 33 us; skew 500 ppm
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:30.186733 7488 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:30.187423 7488 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:30.187486 7488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:30.187664 7488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:30.189510 7488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "58efbbae66b19a52902ab8c77b8a17fc"
server_key_iv: "7fe86ad535feb75c8bb8f648947a0dbc"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.190022 7488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "58efbbae66b19a52902ab8c77b8a17fc"
server_key_iv: "7fe86ad535feb75c8bb8f648947a0dbc"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.193459 7488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:16:30.195652 7515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.196781 7488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:30.196883 7488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "58efbbae66b19a52902ab8c77b8a17fc"
server_key_iv: "7fe86ad535feb75c8bb8f648947a0dbc"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.196981 7488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:30.233146 7488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:30.235755 7488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:30.235925 7488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:30.236445 7488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:30.237226 7488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:30.237273 7488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.237324 7488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:30.237350 7488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.245424 7488 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41927
I20260812 11:16:30.245452 7765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41927 every 8 connection(s)
I20260812 11:16:30.245945 7488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:30.249651 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7488
I20260812 11:16:30.249771 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:30.250043 31742 external_mini_cluster.cc:1428] Setting key 72c591844c9bb078ba0092ed51a03dd6
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:30.258828 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.247562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:42399 (local address 127.30.255.190:40875)
0812 11:16:30.247684 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.247688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.248669 (+ 981us) server_negotiation.cc:407] Connection header received
0812 11:16:30.249482 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.249486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.249541 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.249619 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:30.250822 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.251550 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.252662 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.252972 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.253074 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.255546 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:30.255549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:30.255551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:30.255576 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:30.256875 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.257429 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.257431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.257433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.257505 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.257840 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.257842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.257843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.258018 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:30.258090 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.258617 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.258713 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:16:30.259900 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.247875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40875 (local address 127.30.255.129:42399)
0812 11:16:30.248538 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.248575 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.249273 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.249786 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.249794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.250169 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:30.250697 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.250713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.251670 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.251672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.252541 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.252550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.253092 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.253094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.253237 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.253696 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:30.253715 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:30.255387 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:30.256990 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.256996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.257010 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.257341 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.257582 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.257585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.257587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.257760 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.258111 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:30.258116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:30.258398 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.258598 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.258914 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:30.261059 7768 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40875
I20260812 11:16:30.261261 7768 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:30.261698 7768 heartbeater.cc:511] Master 127.30.255.190:40875 requested a full tablet report, sending...
I20260812 11:16:30.263077 7360 ts_manager.cc:194] Registered new tserver with Master: e0f4e7c4a8f54c598a78d8af11aa7e20 (127.30.255.129:41927)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:30.264189 7360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:42399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:30.278215 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:30.383555 7783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:30.383777 7783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:30.383836 7783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:30.387708 7783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:30.387756 7783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:30.387828 7783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:30.392733 7783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:30.393603 7783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:30.394479 7783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:30.402971 7792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.403023 7794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.402958 7796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:30.403112 7783 server_base.cc:1034] running on GCE node
I20260812 11:16:30.403632 7783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:30.404160 7783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:30.405313 7783 hybrid_clock.cc:648] HybridClock initialized: now 1786533390405297 us; error 30 us; skew 500 ppm
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:30.407833 7783 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:30.408470 7783 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:30.408535 7783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:30.408715 7783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:30.410660 7783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "4ab2583849e44253b3ef94716085d517"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "f0b76df05dce118e6852ce43f300f9de"
server_key_iv: "58ce7c986087bd39176eca52b3bb0634"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.411237 7783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "4ab2583849e44253b3ef94716085d517"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "f0b76df05dce118e6852ce43f300f9de"
server_key_iv: "58ce7c986087bd39176eca52b3bb0634"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.414651 7783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:30.416816 7806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.417907 7783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:30.418021 7783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "4ab2583849e44253b3ef94716085d517"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "f0b76df05dce118e6852ce43f300f9de"
server_key_iv: "58ce7c986087bd39176eca52b3bb0634"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.418107 7783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:30.434422 7783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:30.436913 7783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:30.437067 7783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:30.437564 7783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:30.438308 7783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:30.438354 7783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.438403 7783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:30.438432 7783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.446547 7783 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:41849
I20260812 11:16:30.446611 8025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:41849 every 8 connection(s)
I20260812 11:16:30.447072 7783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:30.453374 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7783
I20260812 11:16:30.453503 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:30.453781 31742 external_mini_cluster.cc:1428] Setting key da9d47da77e43ba44278e469d92ad3f4
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:30.460553 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.448591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:35705 (local address 127.30.255.190:40875)
0812 11:16:30.448705 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.448709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.449466 (+ 757us) server_negotiation.cc:407] Connection header received
0812 11:16:30.450733 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.450736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.450797 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.450869 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:30.452021 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.452556 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.453961 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.454335 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.454430 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.456987 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:30.456990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:30.456992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:30.457017 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:30.458434 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.459011 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.459014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.459015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.459090 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.459461 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.459463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.459464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.459632 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:30.459710 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.460293 (+ 583us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.460413 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:16:30.461561 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.448898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40875 (local address 127.30.255.130:35705)
0812 11:16:30.449316 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.449355 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.450528 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.451046 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.451052 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.451424 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:30.451868 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.451891 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.452676 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.452679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.453830 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.453842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.454489 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.454494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.454643 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.455193 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:30.455209 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:30.456821 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:30.458562 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.458567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.458577 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.458919 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.459179 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.459182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.459183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.459378 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.459729 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:30.459735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:30.460034 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.460260 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.460586 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260812 11:16:30.463346 8026 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40875
I20260812 11:16:30.463557 8026 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:30.463960 8026 heartbeater.cc:511] Master 127.30.255.190:40875 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:30.465006 7360 ts_manager.cc:194] Registered new tserver with Master: 4ab2583849e44253b3ef94716085d517 (127.30.255.130:41849)
I20260812 11:16:30.465570 7360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:35705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:30.479940 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:30.584926 8033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:30.585132 8033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:30.585172 8033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:30.588990 8033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:30.589035 8033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:30.589089 8033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:30.593924 8033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:30.594748 8033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:30.595649 8033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:30.604081 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.604085 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:30.604085 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:30.604331 8033 server_base.cc:1034] running on GCE node
I20260812 11:16:30.604763 8033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:30.605319 8033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:30.606473 8033 hybrid_clock.cc:648] HybridClock initialized: now 1786533390606459 us; error 22 us; skew 500 ppm
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:30.609009 8033 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:30.609694 8033 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:30.609762 8033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:30.609935 8033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:30.611939 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "7dc13d32141a4dea8d6b65d7090dc940"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "585bb26a1831628c0ba37650a78cd1bd"
server_key_iv: "3d618f1285ff2165810022fc36dbdc84"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.612468 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "7dc13d32141a4dea8d6b65d7090dc940"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "585bb26a1831628c0ba37650a78cd1bd"
server_key_iv: "3d618f1285ff2165810022fc36dbdc84"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.615871 8033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:16:30.618108 8057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.619199 8033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:30.619323 8033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "7dc13d32141a4dea8d6b65d7090dc940"
format_stamp: "Formatted at 2026-08-12 11:16:30 on dist-test-slave-gjrz"
server_key: "585bb26a1831628c0ba37650a78cd1bd"
server_key_iv: "3d618f1285ff2165810022fc36dbdc84"
server_key_version: "encryptionkey@0"
I20260812 11:16:30.619416 8033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:30.633437 8033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:30.635998 8033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:30.636135 8033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:30.636585 8033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:30.637315 8033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:30.637359 8033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.637394 8033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:30.637415 8033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:30.645418 8033 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:41855
I20260812 11:16:30.645469 8300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:41855 every 8 connection(s)
I20260812 11:16:30.645923 8033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
Aug 12 11:16:30 dist-test-slave-gjrz krb5kdc[7277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533390, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:30.654982 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8033
I20260812 11:16:30.655133 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:30.655457 31742 external_mini_cluster.cc:1428] Setting key 72719840321b48a621895c7a8da6fb97
I20260812 11:16:30.658272 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.647473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35659 (local address 127.30.255.190:40875)
0812 11:16:30.647627 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.647632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.648391 (+ 759us) server_negotiation.cc:407] Connection header received
0812 11:16:30.649623 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.649626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.649674 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.649744 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:30.650776 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.651340 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.652392 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.652593 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.652670 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.654845 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:30.654849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:30.654852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:30.654880 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:30.656434 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.656888 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.656892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.656893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.656994 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.657298 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.657301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.657303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.657476 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:30.657560 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.657998 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.658117 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:16:30.658915 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.647814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40875 (local address 127.30.255.131:35659)
0812 11:16:30.648254 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.648293 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.649432 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.649884 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.649891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.650217 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:30.650666 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.650680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.651436 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.651449 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.652299 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.652308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.652706 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.652708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.652837 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.653271 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:30.653286 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:30.654656 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:30.656549 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.656553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.656563 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.656795 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.657080 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:30.657083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:30.657084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:30.657211 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:30.657573 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:30.657579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:30.657780 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.657932 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.658208 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":118,"threads_started":1}
I20260812 11:16:30.659855 8303 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40875
I20260812 11:16:30.660056 8303 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:30.660440 8303 heartbeater.cc:511] Master 127.30.255.190:40875 requested a full tablet report, sending...
I20260812 11:16:30.661398 7360 ts_manager.cc:194] Registered new tserver with Master: 7dc13d32141a4dea8d6b65d7090dc940 (127.30.255.131:41855)
I20260812 11:16:30.662431 7360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:35659
I20260812 11:16:30.668857 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:16:30.676719 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.670621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.30.255.190:40875)
0812 11:16:30.670734 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.670739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.670932 (+ 193us) server_negotiation.cc:407] Connection header received
0812 11:16:30.671018 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.671021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.671059 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.671118 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:30.671773 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.672256 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.672988 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.673190 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.673263 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.674331 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:30.674334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:30.674336 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:30.674357 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:30.675463 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.675821 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.675823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.675824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.675887 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:30.676075 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:30.676077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:30.676078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:30.676259 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:30.676357 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.676500 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.676584 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:30.679340 7360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:30.682173 7360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:30.696306 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.690071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:41855 (local address 127.0.0.1:39602)
0812 11:16:30.690486 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.690536 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.690680 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.691906 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.691910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.691942 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:30.692106 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.692114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.693235 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.693238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.694722 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.694733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.695752 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.695863 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.695898 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.696017 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.696175 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":123,"spinlock_wait_cycles":11008,"thread_start_us":58,"threads_started":1}
I20260812 11:16:30.696309 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.691123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.30.255.131:41855)
0812 11:16:30.691531 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.691543 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.691565 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:16:30.691632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.691636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.691770 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.691908 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:30.692240 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.693125 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.694856 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.695630 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695749 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.695872 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.696066 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.696160 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":155,"threads_started":1}
I20260812 11:16:30.696508 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.690898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.30.255.130:41849)
0812 11:16:30.691325 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.691338 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.692399 (+ 1061us) server_negotiation.cc:407] Connection header received
0812 11:16:30.692501 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.692507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.692675 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.692811 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:30.693016 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.694009 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695081 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.695702 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695815 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.695933 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.696255 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.696356 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":106,"threads_started":1}
I20260812 11:16:30.696776 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.690153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41927 (local address 127.0.0.1:35160)
0812 11:16:30.691341 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.691355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.691449 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.691896 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.691902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.691926 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:30.692084 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.692091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.693426 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.693429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.694722 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.694733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.696396 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.696400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.696483 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.696505 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.696580 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.696650 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":102,"spinlock_wait_cycles":2432,"thread_start_us":93,"threads_started":1}
I20260812 11:16:30.696906 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.690710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.30.255.129:41927)
0812 11:16:30.691050 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.691062 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.691530 (+ 468us) server_negotiation.cc:407] Connection header received
0812 11:16:30.691600 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.691604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.691750 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.691869 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:30.692212 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.693274 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695347 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.696105 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.696225 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.696335 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.696663 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.696761 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":134,"threads_started":1}
I20260812 11:16:30.697167 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.690725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:41849 (local address 127.0.0.1:57466)
0812 11:16:30.692131 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.692142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.692393 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.692799 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.692802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.692819 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:30.692927 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.692932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.694166 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.694170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.694967 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.694977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.695957 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.695961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.696059 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.696110 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.696970 (+ 860us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.697047 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":8,"thread_start_us":80,"threads_started":1}
I20260812 11:16:30.699833 8165 tablet_service.cc:1467] Processing CreateTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e (DEFAULT_TABLE table=test-table [id=a03078883a654f0fb24c3cbc461af3b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:30.699833 7629 tablet_service.cc:1467] Processing CreateTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e (DEFAULT_TABLE table=test-table [id=a03078883a654f0fb24c3cbc461af3b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:30.700345 7893 tablet_service.cc:1467] Processing CreateTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e (DEFAULT_TABLE table=test-table [id=a03078883a654f0fb24c3cbc461af3b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:30.700977 7629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c9869c64964cc6ac2a5c6eea39323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:30.700982 8165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c9869c64964cc6ac2a5c6eea39323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:30.701251 7893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c9869c64964cc6ac2a5c6eea39323e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:30.707058 8335 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Bootstrap starting.
I20260812 11:16:30.708554 8337 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Bootstrap starting.
I20260812 11:16:30.708554 8341 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Bootstrap starting.
I20260812 11:16:30.709254 8335 tablet_bootstrap.cc:654] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:30.710335 8335 log.cc:824] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:30.710646 8337 tablet_bootstrap.cc:654] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:30.710642 8341 tablet_bootstrap.cc:654] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:30.711880 8337 log.cc:824] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:30.711880 8341 log.cc:824] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:30.712646 8335 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: No bootstrap required, opened a new log
I20260812 11:16:30.712826 8335 ts_tablet_manager.cc:1397] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:16:30.713757 8337 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: No bootstrap required, opened a new log
I20260812 11:16:30.713757 8341 tablet_bootstrap.cc:492] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: No bootstrap required, opened a new log
I20260812 11:16:30.713940 8337 ts_tablet_manager.cc:1397] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:16:30.713940 8341 ts_tablet_manager.cc:1397] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 11:16:30.715950 8335 raft_consensus.cc:348] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.716159 8335 raft_consensus.cc:374] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:30.716238 8335 raft_consensus.cc:729] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab2583849e44253b3ef94716085d517, State: Initialized, Role: FOLLOWER
I20260812 11:16:30.716636 8335 consensus_queue.cc:260] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.717226 8026 heartbeater.cc:503] Master 127.30.255.190:40875 was elected leader, sending a full tablet report...
I20260812 11:16:30.717491 8335 ts_tablet_manager.cc:1428] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:16:30.717579 8341 raft_consensus.cc:348] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.717839 8341 raft_consensus.cc:374] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:30.717897 8341 raft_consensus.cc:729] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc13d32141a4dea8d6b65d7090dc940, State: Initialized, Role: FOLLOWER
I20260812 11:16:30.718318 8341 consensus_queue.cc:260] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.718760 8337 raft_consensus.cc:348] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.719002 8337 raft_consensus.cc:374] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:30.719092 8337 raft_consensus.cc:729] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f4e7c4a8f54c598a78d8af11aa7e20, State: Initialized, Role: FOLLOWER
I20260812 11:16:30.719340 8341 ts_tablet_manager.cc:1428] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260812 11:16:30.719647 8303 heartbeater.cc:503] Master 127.30.255.190:40875 was elected leader, sending a full tablet report...
I20260812 11:16:30.720386 8337 consensus_queue.cc:260] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.721186 8337 ts_tablet_manager.cc:1428] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 11:16:30.721401 7768 heartbeater.cc:503] Master 127.30.255.190:40875 was elected leader, sending a full tablet report...
W20260812 11:16:30.746865 7769 tablet.cc:2369] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:30.778000 8346 raft_consensus.cc:482] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:30.778200 8346 raft_consensus.cc:504] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.779341 8346 leader_election.cc:288] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dc13d32141a4dea8d6b65d7090dc940 (127.30.255.131:41855), e0f4e7c4a8f54c598a78d8af11aa7e20 (127.30.255.129:41927)
I20260812 11:16:30.783806 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.779516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:41855 (local address 127.30.255.130:41429)
0812 11:16:30.779722 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.779737 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.779823 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.780038 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.780041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.780055 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:30.780178 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.780184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.781174 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.781177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.782577 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.782587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.783449 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.783452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.783539 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.783560 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.783618 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.783670 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":84}
I20260812 11:16:30.783880 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.779679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:41429 (local address 127.30.255.131:41855)
0812 11:16:30.779814 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.779818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.779834 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:16:30.779906 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.779908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.779947 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.780016 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:30.780291 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.781063 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.782694 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.783317 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.783427 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.783479 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.783671 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.783754 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 11:16:30.784217 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.779642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:54583 (local address 127.30.255.129:41927)
0812 11:16:30.779746 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.779751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.780026 (+ 275us) server_negotiation.cc:407] Connection header received
0812 11:16:30.780115 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.780118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.780158 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.780215 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:30.780442 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.781147 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.782693 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.783317 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.783427 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.783479 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.784019 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.784095 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:16:30.784529 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.779589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41927 (local address 127.30.255.130:54583)
0812 11:16:30.779940 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.779950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:30.780037 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:30.780244 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:30.780247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:30.780265 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:30.780355 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.780360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.781236 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.781239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.782576 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:30.782587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.783758 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.783762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:30.783868 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.783891 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:30.784346 (+ 455us) client_negotiation.cc:240] Negotiation successful
0812 11:16:30.784425 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":31,"thread_start_us":103,"threads_started":1}
I20260812 11:16:30.784480 8205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c9869c64964cc6ac2a5c6eea39323e" candidate_uuid: "4ab2583849e44253b3ef94716085d517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc13d32141a4dea8d6b65d7090dc940" is_pre_election: true
I20260812 11:16:30.784765 8205 raft_consensus.cc:2454] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ab2583849e44253b3ef94716085d517 in term 0.
I20260812 11:16:30.785077 7671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c9869c64964cc6ac2a5c6eea39323e" candidate_uuid: "4ab2583849e44253b3ef94716085d517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" is_pre_election: true
I20260812 11:16:30.785231 7807 leader_election.cc:302] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ab2583849e44253b3ef94716085d517, 7dc13d32141a4dea8d6b65d7090dc940; no voters:
I20260812 11:16:30.785365 7671 raft_consensus.cc:2454] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ab2583849e44253b3ef94716085d517 in term 0.
I20260812 11:16:30.785491 8346 raft_consensus.cc:2781] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:30.785558 8346 raft_consensus.cc:482] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:30.785585 8346 raft_consensus.cc:3037] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:30.786554 8346 raft_consensus.cc:504] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.786836 8346 leader_election.cc:288] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [CANDIDATE]: Term 1 election: Requested vote from peers 7dc13d32141a4dea8d6b65d7090dc940 (127.30.255.131:41855), e0f4e7c4a8f54c598a78d8af11aa7e20 (127.30.255.129:41927)
I20260812 11:16:30.787150 8205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c9869c64964cc6ac2a5c6eea39323e" candidate_uuid: "4ab2583849e44253b3ef94716085d517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc13d32141a4dea8d6b65d7090dc940"
I20260812 11:16:30.787150 7671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c9869c64964cc6ac2a5c6eea39323e" candidate_uuid: "4ab2583849e44253b3ef94716085d517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20"
I20260812 11:16:30.787295 7671 raft_consensus.cc:3037] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:30.787284 8205 raft_consensus.cc:3037] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:30.788502 8205 raft_consensus.cc:2454] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ab2583849e44253b3ef94716085d517 in term 1.
I20260812 11:16:30.788502 7671 raft_consensus.cc:2454] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ab2583849e44253b3ef94716085d517 in term 1.
I20260812 11:16:30.788790 7807 leader_election.cc:302] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ab2583849e44253b3ef94716085d517, 7dc13d32141a4dea8d6b65d7090dc940; no voters:
I20260812 11:16:30.788938 8346 raft_consensus.cc:2781] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:30.789139 8346 raft_consensus.cc:686] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 1 LEADER]: Becoming Leader. State: Replica: 4ab2583849e44253b3ef94716085d517, State: Running, Role: LEADER
I20260812 11:16:30.789422 8346 consensus_queue.cc:237] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } }
I20260812 11:16:30.792223 7359 catalog_manager.cc:5495] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ab2583849e44253b3ef94716085d517 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "4ab2583849e44253b3ef94716085d517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ab2583849e44253b3ef94716085d517" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41849 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:30.815012 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.811636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.30.255.130:41849)
0812 11:16:30.811753 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:30.811757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:30.811772 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:16:30.811939 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:30.811942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:30.811990 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:30.812076 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:30.812340 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.812828 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.813671 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:30.813894 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:30.813979 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:30.814281 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:30.814662 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:30.814768 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:30.814841 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 11:16:30.814894 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:30.820188 8205 raft_consensus.cc:1261] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 1 FOLLOWER]: Refusing update from remote peer 4ab2583849e44253b3ef94716085d517: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:30.820188 7671 raft_consensus.cc:1261] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 1 FOLLOWER]: Refusing update from remote peer 4ab2583849e44253b3ef94716085d517: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:30.820834 8352 consensus_queue.cc:1035] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc13d32141a4dea8d6b65d7090dc940" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:30.820998 8346 consensus_queue.cc:1035] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f4e7c4a8f54c598a78d8af11aa7e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:30.828068 8361 mvcc.cc:205] Tried to move back new op lower bound from 7317640768793063424 to 7317640768675872768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640768793063424}]
I20260812 11:16:30.830092 8359 mvcc.cc:205] Tried to move back new op lower bound from 7317640768793063424 to 7317640768675872768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640768793063424}]
I20260812 11:16:30.837978 7359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:30.838107 7359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:30.840315 7359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 870a980c28534978a36c36cb256ff0a6: Sending DeleteTablet for 3 replicas of tablet a1c9869c64964cc6ac2a5c6eea39323e
I20260812 11:16:30.841109 7629 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:30 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:35160
I20260812 11:16:30.841176 8165 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:30 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:39602
I20260812 11:16:30.841279 7893 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c9869c64964cc6ac2a5c6eea39323e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:30 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:57466
I20260812 11:16:30.841665 8382 tablet_replica.cc:330] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: stopping tablet replica
I20260812 11:16:30.841760 8377 tablet_replica.cc:330] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20: stopping tablet replica
I20260812 11:16:30.841996 8382 raft_consensus.cc:2229] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:30.842060 8377 raft_consensus.cc:2229] T a1c9869c64964cc6ac2a5c6eea39323e P e0f4e7c4a8f54c598a78d8af11aa7e20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:30.842233 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7488
I20260812 11:16:30.842365 8382 raft_consensus.cc:2258] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:30.851425 8378 tablet_replica.cc:330] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: stopping tablet replica
I20260812 11:16:30.851707 8378 raft_consensus.cc:2229] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:30.851953 8378 raft_consensus.cc:2258] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:30.866473 8382 ts_tablet_manager.cc:1905] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:30.869735 8382 ts_tablet_manager.cc:1918] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:30.869853 8382 log.cc:1196] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/a1c9869c64964cc6ac2a5c6eea39323e
I20260812 11:16:30.870200 8382 ts_tablet_manager.cc:1939] T a1c9869c64964cc6ac2a5c6eea39323e P 4ab2583849e44253b3ef94716085d517: Deleting consensus metadata
I20260812 11:16:30.871760 7334 catalog_manager.cc:4841] TS 4ab2583849e44253b3ef94716085d517 (127.30.255.130:41849): tablet a1c9869c64964cc6ac2a5c6eea39323e (table test-table [id=a03078883a654f0fb24c3cbc461af3b5]) successfully deleted
I20260812 11:16:30.878413 8378 ts_tablet_manager.cc:1905] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:30.880549 8378 ts_tablet_manager.cc:1918] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:30.880640 8378 log.cc:1196] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/a1c9869c64964cc6ac2a5c6eea39323e
I20260812 11:16:30.880936 8378 ts_tablet_manager.cc:1939] T a1c9869c64964cc6ac2a5c6eea39323e P 7dc13d32141a4dea8d6b65d7090dc940: Deleting consensus metadata
I20260812 11:16:30.881928 7334 catalog_manager.cc:4841] TS 7dc13d32141a4dea8d6b65d7090dc940 (127.30.255.131:41855): tablet a1c9869c64964cc6ac2a5c6eea39323e (table test-table [id=a03078883a654f0fb24c3cbc461af3b5]) successfully deleted
W20260812 11:16:30.884449 7337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:41927 (error 108)
I20260812 11:16:30.885257 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7783
I20260812 11:16:30.885428 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:30.885138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41927 (local address 127.0.0.1:35170)
0812 11:16:30.885276 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:30.885345 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:16:30.891031 7337 catalog_manager.cc:4584] TS e0f4e7c4a8f54c598a78d8af11aa7e20 (127.30.255.129:41927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1c9869c64964cc6ac2a5c6eea39323e: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41927: connect: Connection refused (error 111)
I20260812 11:16:30.896119 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8033
I20260812 11:16:30.910185 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7300
2026-08-12T11:16:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2472 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T11:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:30Z Disabled control of system clock
I20260812 11:16:30.946671 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43929
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 11:16:31.053301 8391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:31.053534 8391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:31.053584 8391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:31.057461 8391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:31.057509 8391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:31.057533 8391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:31.057554 8391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:31.057578 8391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:31.062124 8391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:31.063033 8391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:31.064307 8391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:31.071873 8408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.071938 8416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.071961 8406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.072113 8391 server_base.cc:1034] running on GCE node
I20260812 11:16:31.072626 8391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:31.073580 8391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:31.074738 8391 hybrid_clock.cc:648] HybridClock initialized: now 1786533391074724 us; error 27 us; skew 500 ppm
I20260812 11:16:31.076438 8391 webserver.cc:460] Webserver started at http://127.30.255.190:46063/ using document root <none> and password file <none>
I20260812 11:16:31.076947 8391 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:31.077020 8391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:31.077216 8391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:31.079083 8391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "6df5c74463d942128e00d6893cd9a8e7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "37ff9465fdb099513d994214bb450fe9"
server_key_iv: "a18b88696f045c3249b4aa09badffb4d"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.079648 8391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "6df5c74463d942128e00d6893cd9a8e7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "37ff9465fdb099513d994214bb450fe9"
server_key_iv: "a18b88696f045c3249b4aa09badffb4d"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.083302 8391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:31.085474 8429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.086544 8391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:31.086654 8391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "6df5c74463d942128e00d6893cd9a8e7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "37ff9465fdb099513d994214bb450fe9"
server_key_iv: "a18b88696f045c3249b4aa09badffb4d"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.086740 8391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:31.090950 8391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:31.091523 8391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:31.091668 8391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:31.097783 8391 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:43929
I20260812 11:16:31.097805 8545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:43929 every 8 connection(s)
I20260812 11:16:31.098577 8391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:31.101666 8546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:31.101928 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8391
I20260812 11:16:31.102042 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:31.102313 31742 external_mini_cluster.cc:1428] Setting key 1dd5be4fd79ab37b17b3683e916f25c3
I20260812 11:16:31.107592 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.103716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.30.255.190:43929)
0812 11:16:31.104320 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.104337 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.104369 (+ 32us) server_negotiation.cc:407] Connection header received
0812 11:16:31.104993 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.105009 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.105320 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.105588 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.106390 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.106395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.106406 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.106416 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.106468 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.106491 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.106595 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.106709 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.106917 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":144,"threads_started":1}
I20260812 11:16:31.107632 8546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: Bootstrap starting.
I20260812 11:16:31.109824 8546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:31.110495 8546 log.cc:824] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:31.112208 8546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: No bootstrap required, opened a new log
I20260812 11:16:31.115481 8546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } }
I20260812 11:16:31.115692 8546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:31.115756 8546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df5c74463d942128e00d6893cd9a8e7, State: Initialized, Role: FOLLOWER
I20260812 11:16:31.116165 8546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } }
I20260812 11:16:31.116295 8546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:31.116360 8546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:31.116456 8546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:31.117221 8546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } }
I20260812 11:16:31.117524 8546 leader_election.cc:302] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df5c74463d942128e00d6893cd9a8e7; no voters:
I20260812 11:16:31.117812 8546 leader_election.cc:288] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:31.117889 8560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:31.118104 8560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [term 1 LEADER]: Becoming Leader. State: Replica: 6df5c74463d942128e00d6893cd9a8e7, State: Running, Role: LEADER
I20260812 11:16:31.118407 8560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } }
I20260812 11:16:31.118882 8546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:31.119925 8564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df5c74463d942128e00d6893cd9a8e7. Latest consensus state: current_term: 1 leader_uuid: "6df5c74463d942128e00d6893cd9a8e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } } }
I20260812 11:16:31.120067 8564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:31.120381 8561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6df5c74463d942128e00d6893cd9a8e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df5c74463d942128e00d6893cd9a8e7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43929 } } }
I20260812 11:16:31.120453 8575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:31.120536 8561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:31.123644 8575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:31.128698 8575 catalog_manager.cc:1347] Generated new cluster ID: fdaafe2e49bd4e81b5e971fbe53ffa53
I20260812 11:16:31.128756 8575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:31.155920 8575 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:31.156247 8575 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:31.165022 8575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: Generated new TSK 0
I20260812 11:16:31.165714 8575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:16:31.179463 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--builtin_ntp_servers=127.30.255.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 11:16:31.287873 8601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:31.288115 8601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:31.288169 8601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:31.292203 8601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:31.292250 8601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:31.292323 8601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:31.296890 8601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:31.297823 8601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:31.299098 8601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:31.308307 8612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.308467 8601 server_base.cc:1034] running on GCE node
W20260812 11:16:31.308331 8614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.308370 8620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.308964 8601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:31.309504 8601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:31.310667 8601 hybrid_clock.cc:648] HybridClock initialized: now 1786533391310651 us; error 29 us; skew 500 ppm
I20260812 11:16:31.312336 8601 webserver.cc:460] Webserver started at http://127.30.255.129:42675/ using document root <none> and password file <none>
I20260812 11:16:31.312860 8601 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:31.312925 8601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:31.313108 8601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:31.314990 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "6fc99d281a0f4d568ab5024013ffd495"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "63cae292f092beb91b0248a8a4012304"
server_key_iv: "cec9ba3a36f361871f7c56fd8cf78cdd"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.315542 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "6fc99d281a0f4d568ab5024013ffd495"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "63cae292f092beb91b0248a8a4012304"
server_key_iv: "cec9ba3a36f361871f7c56fd8cf78cdd"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.318989 8601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:16:31.321396 8631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.322508 8601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:31.322621 8601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "6fc99d281a0f4d568ab5024013ffd495"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "63cae292f092beb91b0248a8a4012304"
server_key_iv: "cec9ba3a36f361871f7c56fd8cf78cdd"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.322711 8601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:31.326990 8601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:31.327577 8601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:31.327729 8601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:31.328222 8601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:31.329043 8601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:31.329092 8601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.329141 8601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:31.329170 8601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.336887 8601 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:39663
I20260812 11:16:31.336939 8878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:39663 every 8 connection(s)
I20260812 11:16:31.337670 8601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:31.343225 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.339249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:53961 (local address 127.30.255.190:43929)
0812 11:16:31.339392 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.339395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.340376 (+ 981us) server_negotiation.cc:407] Connection header received
0812 11:16:31.341671 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.341675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.341723 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.341800 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.342785 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.342788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.342791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.342806 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.342824 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.342832 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.342909 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.343038 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.343092 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 11:16:31.343986 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.339626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43929 (local address 127.30.255.129:53961)
0812 11:16:31.340232 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.340270 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.341457 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.341971 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.341978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.342061 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.342529 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.342531 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.342574 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.342638 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.342902 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.342911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.342917 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.343070 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.343320 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":116,"threads_started":1}
I20260812 11:16:31.345059 8882 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43929
I20260812 11:16:31.345268 8882 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:31.345620 8882 heartbeater.cc:511] Master 127.30.255.190:43929 requested a full tablet report, sending...
I20260812 11:16:31.345870 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8601
I20260812 11:16:31.345985 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:31.346263 31742 external_mini_cluster.cc:1428] Setting key 49e0c8b8dab89493312862828e2b092e
I20260812 11:16:31.347122 8467 ts_manager.cc:194] Registered new tserver with Master: 6fc99d281a0f4d568ab5024013ffd495 (127.30.255.129:39663)
I20260812 11:16:31.348181 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--builtin_ntp_servers=127.30.255.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 11:16:31.454623 8887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:31.454838 8887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:31.454874 8887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:31.458729 8887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:31.458775 8887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:31.458825 8887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:31.463302 8887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:31.464221 8887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:31.465473 8887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:31.474418 8896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.474411 8894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.474470 8887 server_base.cc:1034] running on GCE node
W20260812 11:16:31.474423 8898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.475018 8887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:31.475612 8887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:31.476761 8887 hybrid_clock.cc:648] HybridClock initialized: now 1786533391476741 us; error 37 us; skew 500 ppm
I20260812 11:16:31.478422 8887 webserver.cc:460] Webserver started at http://127.30.255.130:42825/ using document root <none> and password file <none>
I20260812 11:16:31.478928 8887 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:31.478993 8887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:31.479179 8887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:31.481051 8887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "4ebacd69cc534d58b82b8c26955e38b9"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "ae77e3cf7097eb86622e91b97728c582"
server_key_iv: "c8bc5df8b29cacdccad344fc1444c5dc"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.481570 8887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "4ebacd69cc534d58b82b8c26955e38b9"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "ae77e3cf7097eb86622e91b97728c582"
server_key_iv: "c8bc5df8b29cacdccad344fc1444c5dc"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.485040 8887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:31.487311 8907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.488381 8887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:31.488493 8887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "4ebacd69cc534d58b82b8c26955e38b9"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "ae77e3cf7097eb86622e91b97728c582"
server_key_iv: "c8bc5df8b29cacdccad344fc1444c5dc"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.488582 8887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:31.493656 8887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:31.494330 8887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:31.494539 8887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:31.495113 8887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:31.496166 8887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:31.496228 8887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.496323 8887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:31.496358 8887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.507534 8887 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:41577
I20260812 11:16:31.507575 9143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:41577 every 8 connection(s)
I20260812 11:16:31.508301 8887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:31.513909 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.509936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:34227 (local address 127.30.255.190:43929)
0812 11:16:31.510072 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.510076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.511052 (+ 976us) server_negotiation.cc:407] Connection header received
0812 11:16:31.512373 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.512377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.512430 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.512508 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.513486 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.513490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.513493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.513505 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.513525 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.513533 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.513608 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.513735 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.513779 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:31.514164 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8887
I20260812 11:16:31.514277 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:31.514526 31742 external_mini_cluster.cc:1428] Setting key 845dc9e55abdc1ac4804bb935d02efa8
I20260812 11:16:31.514760 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.510266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43929 (local address 127.30.255.130:34227)
0812 11:16:31.510899 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.510936 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.512172 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.512693 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.512699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.512780 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.513243 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.513244 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.513278 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.513343 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.513602 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.513610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.513616 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.513766 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.514020 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260812 11:16:31.515832 9144 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43929
I20260812 11:16:31.516036 9144 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:31.516395 9144 heartbeater.cc:511] Master 127.30.255.190:43929 requested a full tablet report, sending...
I20260812 11:16:31.516491 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--builtin_ntp_servers=127.30.255.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 11:16:31.517354 8467 ts_manager.cc:194] Registered new tserver with Master: 4ebacd69cc534d58b82b8c26955e38b9 (127.30.255.130:41577)
W20260812 11:16:31.622978 9154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:31.623191 9154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:31.623227 9154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:31.627213 9154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:31.627282 9154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:31.627344 9154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:31.631834 9154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:31.632717 9154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:31.633929 9154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:31.642627 9178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.642627 9166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:31.642681 9165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:31.642815 9154 server_base.cc:1034] running on GCE node
I20260812 11:16:31.643304 9154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:31.643874 9154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:31.645063 9154 hybrid_clock.cc:648] HybridClock initialized: now 1786533391645043 us; error 32 us; skew 500 ppm
I20260812 11:16:31.646714 9154 webserver.cc:460] Webserver started at http://127.30.255.131:32965/ using document root <none> and password file <none>
I20260812 11:16:31.647218 9154 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:31.647313 9154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:31.647523 9154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:31.649410 9154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "d4ee7a0d17b14a67b9536ab3952252b7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "d301890fca2f3bd5b2bbe65347711af9"
server_key_iv: "30611829ca3db5cd6477d2f5e0d82436"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.649930 9154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "d4ee7a0d17b14a67b9536ab3952252b7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "d301890fca2f3bd5b2bbe65347711af9"
server_key_iv: "30611829ca3db5cd6477d2f5e0d82436"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.653385 9154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:31.655795 9191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.656911 9154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:31.657104 9154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "d4ee7a0d17b14a67b9536ab3952252b7"
format_stamp: "Formatted at 2026-08-12 11:16:31 on dist-test-slave-gjrz"
server_key: "d301890fca2f3bd5b2bbe65347711af9"
server_key_iv: "30611829ca3db5cd6477d2f5e0d82436"
server_key_version: "encryptionkey@0"
I20260812 11:16:31.657220 9154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:31.661525 9154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:31.662068 9154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:31.662216 9154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:31.662742 9154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:31.663607 9154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:31.663654 9154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.663703 9154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:31.663731 9154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:31.672492 9154 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:38973
I20260812 11:16:31.672636 9452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:38973 every 8 connection(s)
I20260812 11:16:31.673461 9154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:31.680497 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.675324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:53283 (local address 127.30.255.190:43929)
0812 11:16:31.675452 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.675456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.676629 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 11:16:31.678324 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.678328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.678372 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.678425 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.679953 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.679957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.679960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.679974 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.679998 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.680008 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.680081 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.680307 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.680356 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:31.681943 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.675730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43929 (local address 127.30.255.131:53283)
0812 11:16:31.676469 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.676519 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.678066 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.678843 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.678852 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.678964 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.679624 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.679626 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.679680 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.679772 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.680152 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.680160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.680165 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.680764 (+ 599us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.681003 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":113,"threads_started":1}
I20260812 11:16:31.686124 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9154
I20260812 11:16:31.686242 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:31.686539 31742 external_mini_cluster.cc:1428] Setting key f92ba325e00511ff9891cc796d5b30d3
I20260812 11:16:31.687388 9453 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43929
I20260812 11:16:31.687616 9453 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:31.688048 9453 heartbeater.cc:511] Master 127.30.255.190:43929 requested a full tablet report, sending...
I20260812 11:16:31.689072 8467 ts_manager.cc:194] Registered new tserver with Master: d4ee7a0d17b14a67b9536ab3952252b7 (127.30.255.131:38973)
I20260812 11:16:31.700125 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:16:31.703433 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.701870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.30.255.190:43929)
0812 11:16:31.701986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.701989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.702210 (+ 221us) server_negotiation.cc:407] Connection header received
0812 11:16:31.702313 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.702316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.702351 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.702415 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.703042 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.703046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.703049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.703068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.703088 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.703096 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.703167 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.703248 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.703313 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:31.706028 8467 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:31.708968 8467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:31.719781 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.717199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:39663 (local address 127.0.0.1:34856)
0812 11:16:31.717452 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.717467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.717548 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.718447 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.718453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.718522 (+ 69us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.718970 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.718972 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.718994 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.719061 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.719484 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.719494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.719500 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.719600 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.719663 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":95,"threads_started":1}
I20260812 11:16:31.720458 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.717657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.30.255.129:39663)
0812 11:16:31.718011 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.718023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.718047 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:16:31.718101 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.718107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.718292 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.718437 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.719240 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.719246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.719280 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.719293 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.719339 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.719372 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.720041 (+ 669us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.720151 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.720266 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":141,"threads_started":1}
I20260812 11:16:31.720477 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.716918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38973 (local address 127.0.0.1:35172)
0812 11:16:31.717214 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.717256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.717349 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.718751 (+ 1402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.718755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.718770 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.719088 (+ 318us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.719091 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.719102 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.719131 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.720280 (+ 1149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.720283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.720287 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.720343 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.720376 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260812 11:16:31.721640 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.717580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:41577 (local address 127.0.0.1:46288)
0812 11:16:31.717824 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.717837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.717912 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.720353 (+ 2441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.720356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.720371 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.721018 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.721020 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.721030 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.721056 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.721416 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.721420 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.721423 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.721488 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.721521 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
I20260812 11:16:31.722178 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.716993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.30.255.131:38973)
0812 11:16:31.718320 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.718331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.718351 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:16:31.718394 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.718399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.718562 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.719156 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.719746 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.719751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.719761 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.719771 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.719809 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.719834 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.721910 (+ 2076us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.721959 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.722027 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1236,"thread_start_us":139,"threads_started":1}
I20260812 11:16:31.722281 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.717997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.30.255.130:41577)
0812 11:16:31.719480 (+ 1483us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.719491 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.719513 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:16:31.719563 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.719568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.719721 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.719837 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.721187 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.721193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.721204 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.721215 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.721254 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.721280 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.721365 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.722062 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.722135 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1370,"thread_start_us":117,"threads_started":1}
I20260812 11:16:31.723920 8731 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a (DEFAULT_TABLE table=test-table [id=e4d09304f68f4bd2b82ddc49eda0fd68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:31.725039 8731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9df182fb0244979ebf4cd55722eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:31.724905 9291 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a (DEFAULT_TABLE table=test-table [id=e4d09304f68f4bd2b82ddc49eda0fd68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:31.725026 9007 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a (DEFAULT_TABLE table=test-table [id=e4d09304f68f4bd2b82ddc49eda0fd68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:31.725901 9291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9df182fb0244979ebf4cd55722eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:31.726002 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9df182fb0244979ebf4cd55722eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:31.732739 9488 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Bootstrap starting.
I20260812 11:16:31.734375 9490 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Bootstrap starting.
I20260812 11:16:31.734862 9488 tablet_bootstrap.cc:654] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:31.734941 9489 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Bootstrap starting.
I20260812 11:16:31.735765 9488 log.cc:824] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:31.736474 9490 tablet_bootstrap.cc:654] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:31.737040 9489 tablet_bootstrap.cc:654] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:31.737265 9490 log.cc:824] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:31.737804 9489 log.cc:824] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:31.739380 9488 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: No bootstrap required, opened a new log
I20260812 11:16:31.739594 9488 ts_tablet_manager.cc:1397] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:16:31.739625 9489 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: No bootstrap required, opened a new log
I20260812 11:16:31.740012 9489 ts_tablet_manager.cc:1397] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:16:31.743124 9489 raft_consensus.cc:348] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.743382 9489 raft_consensus.cc:374] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:31.743446 9489 raft_consensus.cc:729] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebacd69cc534d58b82b8c26955e38b9, State: Initialized, Role: FOLLOWER
I20260812 11:16:31.743885 9489 consensus_queue.cc:260] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.744028 9488 raft_consensus.cc:348] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.744334 9488 raft_consensus.cc:374] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:31.744418 9488 raft_consensus.cc:729] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc99d281a0f4d568ab5024013ffd495, State: Initialized, Role: FOLLOWER
I20260812 11:16:31.744894 9490 tablet_bootstrap.cc:492] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: No bootstrap required, opened a new log
I20260812 11:16:31.744964 9489 ts_tablet_manager.cc:1428] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:16:31.745080 9490 ts_tablet_manager.cc:1397] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260812 11:16:31.744997 9488 consensus_queue.cc:260] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.745412 9144 heartbeater.cc:503] Master 127.30.255.190:43929 was elected leader, sending a full tablet report...
I20260812 11:16:31.745934 9488 ts_tablet_manager.cc:1428] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 11:16:31.746239 8882 heartbeater.cc:503] Master 127.30.255.190:43929 was elected leader, sending a full tablet report...
I20260812 11:16:31.749403 9490 raft_consensus.cc:348] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.749655 9490 raft_consensus.cc:374] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:31.749707 9490 raft_consensus.cc:729] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4ee7a0d17b14a67b9536ab3952252b7, State: Initialized, Role: FOLLOWER
I20260812 11:16:31.750138 9490 consensus_queue.cc:260] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.750764 9453 heartbeater.cc:503] Master 127.30.255.190:43929 was elected leader, sending a full tablet report...
I20260812 11:16:31.751077 9490 ts_tablet_manager.cc:1428] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
W20260812 11:16:31.759102 9147 tablet.cc:2369] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:31.838667 8884 tablet.cc:2369] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:31.852495 9502 raft_consensus.cc:482] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:31.852684 9502 raft_consensus.cc:504] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.853752 9502 leader_election.cc:288] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ebacd69cc534d58b82b8c26955e38b9 (127.30.255.130:41577), 6fc99d281a0f4d568ab5024013ffd495 (127.30.255.129:39663)
I20260812 11:16:31.855458 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.853949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:41577 (local address 127.30.255.131:36041)
0812 11:16:31.854114 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.854139 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.854217 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.854439 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.854442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.854456 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.854879 (+ 423us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.854881 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.854894 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.854926 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.855194 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.855197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.855201 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.855297 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.855335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 11:16:31.855489 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.854045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:36041 (local address 127.30.255.130:41577)
0812 11:16:31.854153 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.854158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.854175 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:16:31.854306 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.854309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.854343 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.854407 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.855066 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.855070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.855073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.855087 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.855108 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.855119 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.855203 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.855316 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.855360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:16:31.855562 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.854050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45683 (local address 127.30.255.129:39663)
0812 11:16:31.854170 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.854174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.854383 (+ 209us) server_negotiation.cc:407] Connection header received
0812 11:16:31.854477 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.854481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.854515 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.854579 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.855179 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.855184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.855186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.855202 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.855225 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.855235 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.855350 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.855413 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.855451 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:16:31.855628 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.853971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:39663 (local address 127.30.255.131:45683)
0812 11:16:31.854299 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.854310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:31.854394 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:31.854614 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:31.854617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:31.854631 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:31.855013 (+ 382us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:16:31.855015 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:31.855027 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:16:31.855051 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:31.855334 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:31.855337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:31.855341 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:16:31.855413 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:16:31.855446 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":110,"threads_started":1}
I20260812 11:16:31.856191 8768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9df182fb0244979ebf4cd55722eb7a" candidate_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc99d281a0f4d568ab5024013ffd495" is_pre_election: true
I20260812 11:16:31.856451 8768 raft_consensus.cc:2454] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4ee7a0d17b14a67b9536ab3952252b7 in term 0.
I20260812 11:16:31.856446 9045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9df182fb0244979ebf4cd55722eb7a" candidate_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ebacd69cc534d58b82b8c26955e38b9" is_pre_election: true
I20260812 11:16:31.856683 9045 raft_consensus.cc:2454] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4ee7a0d17b14a67b9536ab3952252b7 in term 0.
I20260812 11:16:31.856845 9195 leader_election.cc:302] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc99d281a0f4d568ab5024013ffd495, d4ee7a0d17b14a67b9536ab3952252b7; no voters:
I20260812 11:16:31.857080 9502 raft_consensus.cc:2781] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:31.857167 9502 raft_consensus.cc:482] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:31.857198 9502 raft_consensus.cc:3037] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:31.858207 9502 raft_consensus.cc:504] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.858523 9502 leader_election.cc:288] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4ebacd69cc534d58b82b8c26955e38b9 (127.30.255.130:41577), 6fc99d281a0f4d568ab5024013ffd495 (127.30.255.129:39663)
I20260812 11:16:31.858839 9045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9df182fb0244979ebf4cd55722eb7a" candidate_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ebacd69cc534d58b82b8c26955e38b9"
I20260812 11:16:31.858839 8768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9df182fb0244979ebf4cd55722eb7a" candidate_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc99d281a0f4d568ab5024013ffd495"
I20260812 11:16:31.858939 8768 raft_consensus.cc:3037] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:31.858948 9045 raft_consensus.cc:3037] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:31.859970 8768 raft_consensus.cc:2454] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4ee7a0d17b14a67b9536ab3952252b7 in term 1.
I20260812 11:16:31.859993 9045 raft_consensus.cc:2454] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4ee7a0d17b14a67b9536ab3952252b7 in term 1.
I20260812 11:16:31.860216 9195 leader_election.cc:302] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc99d281a0f4d568ab5024013ffd495, d4ee7a0d17b14a67b9536ab3952252b7; no voters:
I20260812 11:16:31.860366 9502 raft_consensus.cc:2781] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:31.860561 9502 raft_consensus.cc:686] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 1 LEADER]: Becoming Leader. State: Replica: d4ee7a0d17b14a67b9536ab3952252b7, State: Running, Role: LEADER
I20260812 11:16:31.860884 9502 consensus_queue.cc:237] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } }
I20260812 11:16:31.863740 8467 catalog_manager.cc:5495] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4ee7a0d17b14a67b9536ab3952252b7 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4ee7a0d17b14a67b9536ab3952252b7" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:31.884488 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.883358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.30.255.131:38973)
0812 11:16:31.883461 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:31.883464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:31.883476 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:31.883601 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:31.883603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:31.883655 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:31.883718 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:31.884231 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:31.884234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:31.884236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:31.884246 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:31.884263 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:31.884269 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:31.884338 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:31.884376 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 11:16:31.884404 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:16:31.889533 9045 raft_consensus.cc:1261] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 1 FOLLOWER]: Refusing update from remote peer d4ee7a0d17b14a67b9536ab3952252b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:31.889535 8768 raft_consensus.cc:1261] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 1 FOLLOWER]: Refusing update from remote peer d4ee7a0d17b14a67b9536ab3952252b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:31.890157 9506 consensus_queue.cc:1035] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ebacd69cc534d58b82b8c26955e38b9" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:31.890357 9502 consensus_queue.cc:1035] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc99d281a0f4d568ab5024013ffd495" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:31.897498 9517 mvcc.cc:205] Tried to move back new op lower bound from 7317640773173862400 to 7317640773064663040. Current Snapshot: MvccSnapshot[applied={T|T < 7317640773173862400}]
I20260812 11:16:31.898622 9514 mvcc.cc:205] Tried to move back new op lower bound from 7317640773173862400 to 7317640773064663040. Current Snapshot: MvccSnapshot[applied={T|T < 7317640773173862400 or (T in {7317640773173862400})}]
I20260812 11:16:31.900434 9515 mvcc.cc:205] Tried to move back new op lower bound from 7317640773173862400 to 7317640773064663040. Current Snapshot: MvccSnapshot[applied={T|T < 7317640773173862400}]
I20260812 11:16:31.904606 8466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:31.904733 8466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:31.906854 8466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6df5c74463d942128e00d6893cd9a8e7: Sending DeleteTablet for 3 replicas of tablet 5a9df182fb0244979ebf4cd55722eb7a
I20260812 11:16:31.907689 8731 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:31 UTC) from {username='slave'} at 127.0.0.1:34856
I20260812 11:16:31.907691 9291 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:31 UTC) from {username='slave'} at 127.0.0.1:35172
I20260812 11:16:31.907691 9007 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9df182fb0244979ebf4cd55722eb7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:31 UTC) from {username='slave'} at 127.0.0.1:46288
I20260812 11:16:31.908252 9534 tablet_replica.cc:330] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495: stopping tablet replica
I20260812 11:16:31.908402 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8601
I20260812 11:16:31.908656 9534 raft_consensus.cc:2229] T 5a9df182fb0244979ebf4cd55722eb7a P 6fc99d281a0f4d568ab5024013ffd495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:31.911681 9536 tablet_replica.cc:330] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: stopping tablet replica
I20260812 11:16:31.911907 9536 raft_consensus.cc:2229] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:31.912145 9536 raft_consensus.cc:2258] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:31.918705 9536 ts_tablet_manager.cc:1905] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:31.920660 9537 tablet_replica.cc:330] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: stopping tablet replica
I20260812 11:16:31.920902 9537 raft_consensus.cc:2229] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:31.921128 9537 raft_consensus.cc:2258] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:31.921957 9536 ts_tablet_manager.cc:1918] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:31.922040 9536 log.cc:1196] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/5a9df182fb0244979ebf4cd55722eb7a
I20260812 11:16:31.922288 9536 ts_tablet_manager.cc:1939] T 5a9df182fb0244979ebf4cd55722eb7a P d4ee7a0d17b14a67b9536ab3952252b7: Deleting consensus metadata
I20260812 11:16:31.923287 8433 catalog_manager.cc:4841] TS d4ee7a0d17b14a67b9536ab3952252b7 (127.30.255.131:38973): tablet 5a9df182fb0244979ebf4cd55722eb7a (table test-table [id=e4d09304f68f4bd2b82ddc49eda0fd68]) successfully deleted
I20260812 11:16:31.925539 9537 ts_tablet_manager.cc:1905] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:31.927744 9537 ts_tablet_manager.cc:1918] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:31.927832 9537 log.cc:1196] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/5a9df182fb0244979ebf4cd55722eb7a
I20260812 11:16:31.928085 9537 ts_tablet_manager.cc:1939] T 5a9df182fb0244979ebf4cd55722eb7a P 4ebacd69cc534d58b82b8c26955e38b9: Deleting consensus metadata
I20260812 11:16:31.929044 8433 catalog_manager.cc:4841] TS 4ebacd69cc534d58b82b8c26955e38b9 (127.30.255.130:41577): tablet 5a9df182fb0244979ebf4cd55722eb7a (table test-table [id=e4d09304f68f4bd2b82ddc49eda0fd68]) successfully deleted
W20260812 11:16:31.964035 8437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:39663 (error 108)
I20260812 11:16:31.964205 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8887
I20260812 11:16:31.969218 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:31.968855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:39663 (local address 127.0.0.1:34872)
0812 11:16:31.969049 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:31.969120 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:39663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260812 11:16:31.969398 8437 catalog_manager.cc:4584] TS 6fc99d281a0f4d568ab5024013ffd495 (127.30.255.129:39663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a9df182fb0244979ebf4cd55722eb7a: Network error: Client connection negotiation failed: client connection to 127.30.255.129:39663: connect: Connection refused (error 111)
I20260812 11:16:31.982393 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9154
I20260812 11:16:31.992022 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8391
2026-08-12T11:16:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1083 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 11:16:32.022778 31742 mini_oidc.cc:152] Starting JWKS server
I20260812 11:16:32.024461 31742 webserver.cc:460] Webserver started at https://127.0.0.1:33569/ using document root <none> and password file <none>
I20260812 11:16:32.024606 31742 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:16:32.024840 31742 webserver.cc:460] Webserver started at http://127.0.0.1:35523/ using document root <none> and password file <none>
2026-08-12T11:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:32Z Disabled control of system clock
I20260812 11:16:32.033738 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39331
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:39331
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33569/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/testchainca.cert with env {}
W20260812 11:16:32.139225 9552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:32.139469 9552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:32.139510 9552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:32.143399 9552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:32.143442 9552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:32.143457 9552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:32.143487 9552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:32.143513 9552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:16:32.143539 9552 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:16:32.143564 9552 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33569/jwks.json
I20260812 11:16:32.148681 9552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:39331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33569/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:32.149570 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:32.150476 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:32.158485 9563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:32.158493 9564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.158535 9552 server_base.cc:1034] running on GCE node
W20260812 11:16:32.158504 9566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.159162 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:32.160197 9552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:32.161367 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786533392161343 us; error 32 us; skew 500 ppm
I20260812 11:16:32.163147 9552 webserver.cc:460] Webserver started at http://127.30.255.190:40665/ using document root <none> and password file <none>
I20260812 11:16:32.163690 9552 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:32.163755 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:32.163951 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:32.165823 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "fd73feed7a844e47857c201177c3004b"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "a0eca79a606984166fa321ddd40a85ee"
server_key_iv: "045995cac4d27b0de47b15dfbacf09f2"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.166383 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "fd73feed7a844e47857c201177c3004b"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "a0eca79a606984166fa321ddd40a85ee"
server_key_iv: "045995cac4d27b0de47b15dfbacf09f2"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.169966 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:16:32.172184 9576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:32.173278 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:32.173398 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "fd73feed7a844e47857c201177c3004b"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "a0eca79a606984166fa321ddd40a85ee"
server_key_iv: "045995cac4d27b0de47b15dfbacf09f2"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.173489 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:32.184492 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:32.185052 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:32.185210 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:32.191576 9552 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:39331
I20260812 11:16:32.191587 9683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:39331 every 8 connection(s)
I20260812 11:16:32.192394 9552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:32.194954 9684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:32.198390 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9552
I20260812 11:16:32.198514 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:32.198801 31742 external_mini_cluster.cc:1428] Setting key 8ac68db04a43ae3c45890bf7fe20afc4
I20260812 11:16:32.200498 9684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b: Bootstrap starting.
I20260812 11:16:32.202577 9684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:32.203248 9684 log.cc:824] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:32.205194 9684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b: No bootstrap required, opened a new log
I20260812 11:16:32.207876 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.200095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.30.255.190:39331)
0812 11:16:32.200722 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.200737 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.200771 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:16:32.201365 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.201381 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.201754 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.202058 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:32.202482 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.203531 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.204644 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.205101 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.205223 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.205988 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:32.205993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:32.206005 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:32.206351 (+ 346us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:32.206407 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:32.206427 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:32.206498 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:32.206664 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 11:16:32.206907 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":126,"threads_started":1}
I20260812 11:16:32.208319 9684 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } }
I20260812 11:16:32.208516 9684 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:32.208572 9684 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd73feed7a844e47857c201177c3004b, State: Initialized, Role: FOLLOWER
I20260812 11:16:32.208945 9684 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } }
I20260812 11:16:32.209071 9684 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:32.209136 9684 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:32.209231 9684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:32.210071 9684 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } }
I20260812 11:16:32.210367 9684 leader_election.cc:302] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd73feed7a844e47857c201177c3004b; no voters:
I20260812 11:16:32.210422 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:16:32.210677 9684 leader_election.cc:288] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:32.210786 9695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:32.210976 9695 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [term 1 LEADER]: Becoming Leader. State: Replica: fd73feed7a844e47857c201177c3004b, State: Running, Role: LEADER
I20260812 11:16:32.211355 9695 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } }
I20260812 11:16:32.211820 9684 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:32.212921 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd73feed7a844e47857c201177c3004b. Latest consensus state: current_term: 1 leader_uuid: "fd73feed7a844e47857c201177c3004b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } } }
I20260812 11:16:32.213048 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:32.213428 9696 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd73feed7a844e47857c201177c3004b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd73feed7a844e47857c201177c3004b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39331 } } }
I20260812 11:16:32.213567 9696 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:32.213447 9716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:32.216667 9716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:32.221648 9716 catalog_manager.cc:1347] Generated new cluster ID: 4e6cc323edd246db825109db23a81cff
I20260812 11:16:32.221714 9716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:32.230083 9716 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:32.231133 9716 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:32.238190 9716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd73feed7a844e47857c201177c3004b: Generated new TSK 0
I20260812 11:16:32.238837 9716 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 11:16:32.267225 9740 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 11:16:32.267421 9740 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 11:16:32.267940 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.250677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.30.255.190:39331)
0812 11:16:32.250786 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.250791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.250973 (+ 182us) server_negotiation.cc:407] Connection header received
0812 11:16:32.251160 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.251163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.251206 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.251318 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:16:32.251563 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.252071 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.253249 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.253451 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.253515 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.253962 (+ 447us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:16:32.267573 (+ 13611us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 11:16:32.267808 (+ 235us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 11:16:32.268013 9691 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 11:16:32.269008 9739 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:32.250551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39331 (local address 127.0.0.1:49480)
0812 11:16:32.250885 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:32.250899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:32.251025 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:32.251330 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:32.251333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:32.251341 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:16:32.251464 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.251470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.252177 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.252181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.253151 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.253161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.253543 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.253547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.253641 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.253692 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:16:32.267742 (+ 14050us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 11:16:32.267900 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.255.190:39331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.255.190:39331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 11:16:32.269670 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9552
2026-08-12T11:16:32Z chronyd exiting
I20260812 11:16:32.333803 31742 test_util.cc:175] -----------------------------------------------
I20260812 11:16:32.333889 31742 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786533365564394-31742-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 11:16:32.334980 31742 mini_oidc.cc:152] Starting JWKS server
I20260812 11:16:32.335862 31742 webserver.cc:460] Webserver started at https://127.0.0.1:45821/ using document root <none> and password file <none>
I20260812 11:16:32.335953 31742 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:16:32.336207 31742 webserver.cc:460] Webserver started at http://127.0.0.1:34805/ using document root <none> and password file <none>
2026-08-12T11:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:32Z Disabled control of system clock
I20260812 11:16:32.345549 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42211
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:42211
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45821/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/testchainca.cert with env {}
W20260812 11:16:32.452365 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:32.452592 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:32.452627 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:32.456429 9748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:32.456470 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:32.456483 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:32.456502 9748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:32.456517 9748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:16:32.456534 9748 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:16:32.456547 9748 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45821/jwks.json
I20260812 11:16:32.461529 9748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:42211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45821/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:32.462409 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:32.463302 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:32.470978 9762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.471117 9748 server_base.cc:1034] running on GCE node
W20260812 11:16:32.470988 9757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:32.471028 9759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.471778 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:32.472764 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:32.473917 9748 hybrid_clock.cc:648] HybridClock initialized: now 1786533392473904 us; error 24 us; skew 500 ppm
I20260812 11:16:32.475629 9748 webserver.cc:460] Webserver started at http://127.30.255.190:35779/ using document root <none> and password file <none>
I20260812 11:16:32.476135 9748 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:32.476197 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:32.476384 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:32.478255 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "d77709ea9b3a45469e19bd0facebff74"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "9c745d1f0cff7dce6ed8806b0b5121db"
server_key_iv: "9117d26487c723fe8d6c7bbb5666639e"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.478849 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "d77709ea9b3a45469e19bd0facebff74"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "9c745d1f0cff7dce6ed8806b0b5121db"
server_key_iv: "9117d26487c723fe8d6c7bbb5666639e"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.482347 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:16:32.484544 9773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:32.485641 9748 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:32.485750 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "d77709ea9b3a45469e19bd0facebff74"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "9c745d1f0cff7dce6ed8806b0b5121db"
server_key_iv: "9117d26487c723fe8d6c7bbb5666639e"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.485841 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:32.498116 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:32.498658 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:32.498811 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:32.505005 9748 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:42211
I20260812 11:16:32.505002 9882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:42211 every 8 connection(s)
I20260812 11:16:32.505852 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:32.508762 9883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:32.510313 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9748
I20260812 11:16:32.510432 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:32.510673 31742 external_mini_cluster.cc:1428] Setting key b65e773526d557e444f2aa41217b0bf1
I20260812 11:16:32.514669 9883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74: Bootstrap starting.
I20260812 11:16:32.516804 9883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:32.517499 9883 log.cc:824] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:32.519219 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.511955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.30.255.190:42211)
0812 11:16:32.512565 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.512579 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.512613 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:16:32.513250 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.513268 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.513631 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.513921 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:32.514336 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.515356 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.516250 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.516715 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.516859 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.517616 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:32.517622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:32.517640 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:32.518003 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:32.518055 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:32.518075 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:32.518167 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:32.518314 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 11:16:32.518555 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":129,"threads_started":1}
I20260812 11:16:32.519320 9883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74: No bootstrap required, opened a new log
I20260812 11:16:32.521860 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:16:32.523049 9883 raft_consensus.cc:348] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } }
I20260812 11:16:32.523289 9883 raft_consensus.cc:374] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:32.523350 9883 raft_consensus.cc:729] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77709ea9b3a45469e19bd0facebff74, State: Initialized, Role: FOLLOWER
I20260812 11:16:32.523744 9883 consensus_queue.cc:260] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } }
I20260812 11:16:32.523869 9883 raft_consensus.cc:388] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:32.523913 9883 raft_consensus.cc:482] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:32.523980 9883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:32.524729 9883 raft_consensus.cc:504] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } }
I20260812 11:16:32.525007 9883 leader_election.cc:302] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d77709ea9b3a45469e19bd0facebff74; no voters:
I20260812 11:16:32.525252 9883 leader_election.cc:288] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:32.525377 9890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:32.525568 9890 raft_consensus.cc:686] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [term 1 LEADER]: Becoming Leader. State: Replica: d77709ea9b3a45469e19bd0facebff74, State: Running, Role: LEADER
I20260812 11:16:32.525900 9890 consensus_queue.cc:237] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } }
I20260812 11:16:32.526330 9883 sys_catalog.cc:564] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:32.527545 9891 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d77709ea9b3a45469e19bd0facebff74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } } }
I20260812 11:16:32.527688 9891 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:32.528000 9894 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d77709ea9b3a45469e19bd0facebff74. Latest consensus state: current_term: 1 leader_uuid: "d77709ea9b3a45469e19bd0facebff74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77709ea9b3a45469e19bd0facebff74" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42211 } } }
I20260812 11:16:32.528069 9910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:32.528141 9894 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:32.531203 9910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:32.536006 9910 catalog_manager.cc:1347] Generated new cluster ID: 2c030480f81a4144a6adfa71bbba029d
I20260812 11:16:32.536083 9910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:32.547580 9910 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:32.548360 9910 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:32.552202 9910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d77709ea9b3a45469e19bd0facebff74: Generated new TSK 0
I20260812 11:16:32.552760 9910 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 11:16:32.582978 9947 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 11:16:32.583038 9947 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 11:16:32.583685 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.568537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.30.255.190:42211)
0812 11:16:32.568653 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.568658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.568802 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:16:32.568930 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.568933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.568985 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.569084 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:16:32.569286 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.570101 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.570895 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.571247 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.571377 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.571630 (+ 253us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:16:32.583328 (+ 11698us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 11:16:32.583553 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 11:16:32.583746 9887 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 11:16:32.583747 9946 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:32.568448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:42211 (local address 127.0.0.1:48372)
0812 11:16:32.568728 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:32.568740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:32.568833 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:32.569087 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:32.569089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:32.569095 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:16:32.569204 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.569209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.570189 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.570192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.570802 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.570808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.571378 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.571381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.571453 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.571475 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:16:32.583478 (+ 12003us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 11:16:32.583616 (+ 138us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.255.190:42211: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":106,"threads_started":1}
I20260812 11:16:32.584332 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9748
2026-08-12T11:16:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 11:16:32.635329 31742 mini_oidc.cc:152] Starting JWKS server
I20260812 11:16:32.636162 31742 webserver.cc:460] Webserver started at https://127.0.0.1:34135/ using document root <none> and password file <none>
I20260812 11:16:32.636258 31742 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:16:32.636507 31742 webserver.cc:460] Webserver started at http://127.0.0.1:40263/ using document root <none> and password file <none>
2026-08-12T11:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:32Z Disabled control of system clock
I20260812 11:16:32.645618 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42823
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:42823
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34135/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 11:16:32.752204 9961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:32.752427 9961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:32.752465 9961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:32.756362 9961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:32.756414 9961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:32.756428 9961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:32.756457 9961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:32.756484 9961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:16:32.756510 9961 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:16:32.756536 9961 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34135/jwks.json
I20260812 11:16:32.761162 9961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:42823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34135/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:32.762089 9961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:32.763406 9961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:32.771030 9973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:32.771032 9972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.771301 9961 server_base.cc:1034] running on GCE node
W20260812 11:16:32.771370 9979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:32.771938 9961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:32.772971 9961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:32.774125 9961 hybrid_clock.cc:648] HybridClock initialized: now 1786533392774110 us; error 27 us; skew 500 ppm
I20260812 11:16:32.775813 9961 webserver.cc:460] Webserver started at http://127.30.255.190:39005/ using document root <none> and password file <none>
I20260812 11:16:32.776330 9961 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:32.776391 9961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:32.776590 9961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:32.778508 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "162d5b47e51f4675a9eadeb93c6e5962"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "b5a4b54b23ee4d88872ba173bb60efcb"
server_key_iv: "93e3bd6283a406ce5ab2204dbada45a0"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.779037 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "162d5b47e51f4675a9eadeb93c6e5962"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "b5a4b54b23ee4d88872ba173bb60efcb"
server_key_iv: "93e3bd6283a406ce5ab2204dbada45a0"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.782508 9961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:16:32.784752 9990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:32.785854 9961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:32.785964 9961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "162d5b47e51f4675a9eadeb93c6e5962"
format_stamp: "Formatted at 2026-08-12 11:16:32 on dist-test-slave-gjrz"
server_key: "b5a4b54b23ee4d88872ba173bb60efcb"
server_key_iv: "93e3bd6283a406ce5ab2204dbada45a0"
server_key_version: "encryptionkey@0"
I20260812 11:16:32.786053 9961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:32.799160 9961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:32.799742 9961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:32.799890 9961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:32.806166 9961 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:42823
I20260812 11:16:32.806190 10092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:42823 every 8 connection(s)
I20260812 11:16:32.806963 9961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:32.809558 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:32.810532 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9961
I20260812 11:16:32.810668 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:32.811000 31742 external_mini_cluster.cc:1428] Setting key 9f8e9f6109c467a2ad018b59914ac5e1
I20260812 11:16:32.815321 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: Bootstrap starting.
I20260812 11:16:32.817400 10093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:32.818109 10093 log.cc:824] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:32.819429 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.812257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.30.255.190:42823)
0812 11:16:32.812850 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.812864 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.812897 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:16:32.813500 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.813516 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.813886 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.814171 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:32.814583 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.815574 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.816429 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.816891 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.817034 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.817836 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:32.817841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:32.817853 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:32.818203 (+ 350us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:32.818270 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:32.818291 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:32.818372 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:32.818510 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:16:32.818749 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260812 11:16:32.819972 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: No bootstrap required, opened a new log
I20260812 11:16:32.821862 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:16:32.823596 10093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } }
I20260812 11:16:32.823796 10093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:32.823833 10093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162d5b47e51f4675a9eadeb93c6e5962, State: Initialized, Role: FOLLOWER
I20260812 11:16:32.824175 10093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } }
I20260812 11:16:32.824281 10093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:32.824322 10093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:32.824389 10093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:32.825125 10093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } }
I20260812 11:16:32.825407 10093 leader_election.cc:302] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 162d5b47e51f4675a9eadeb93c6e5962; no voters:
I20260812 11:16:32.825630 10093 leader_election.cc:288] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:32.825728 10105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:32.825932 10105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [term 1 LEADER]: Becoming Leader. State: Replica: 162d5b47e51f4675a9eadeb93c6e5962, State: Running, Role: LEADER
I20260812 11:16:32.826262 10105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } }
I20260812 11:16:32.826730 10093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:32.827718 10107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 162d5b47e51f4675a9eadeb93c6e5962. Latest consensus state: current_term: 1 leader_uuid: "162d5b47e51f4675a9eadeb93c6e5962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } } }
I20260812 11:16:32.827891 10107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:32.827929 10106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "162d5b47e51f4675a9eadeb93c6e5962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "162d5b47e51f4675a9eadeb93c6e5962" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42823 } } }
I20260812 11:16:32.828081 10106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962 [sys.catalog]: This master's current role is: LEADER
W20260812 11:16:32.831501 10139 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:32.831585 10139 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:32.831795 10120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:32.832557 10120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:32.837291 10120 catalog_manager.cc:1347] Generated new cluster ID: 59d849524ada4d6c91a55cd7264664b3
I20260812 11:16:32.837349 10120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:32.851016 10120 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:32.851845 10120 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:32.860585 10120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 162d5b47e51f4675a9eadeb93c6e5962: Generated new TSK 0
I20260812 11:16:32.861126 10120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:16:32.876406 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:32.871582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.30.255.190:42823)
0812 11:16:32.871742 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:32.871746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:32.871838 (+ 92us) server_negotiation.cc:407] Connection header received
0812 11:16:32.871984 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:32.871987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:32.872031 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:32.872117 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:16:32.872424 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.872929 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.874202 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.874415 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.874495 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.874826 (+ 331us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:16:32.876081 (+ 1255us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34135/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 11:16:32.876290 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39748: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34135/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 11:16:32.876458 10100 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39748: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34135/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 11:16:32.878269 10157 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:32.871445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:42823 (local address 127.0.0.1:39748)
0812 11:16:32.871752 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:32.871767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:32.871876 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:32.872171 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:32.872175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:32.872185 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:16:32.872306 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.872313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.873051 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.873054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.874077 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:32.874089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:32.874525 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:32.874529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:32.874622 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:32.874649 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:16:32.876230 (+ 1581us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34135/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 11:16:32.878098 (+ 1868us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.255.190:42823: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34135/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1670,"thread_start_us":104,"threads_started":1}
I20260812 11:16:32.878772 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9961
E20260812 11:16:32.896425 10159 webserver.cc:533] Webserver: sslize failed:
E20260812 11:16:32.896495 10159 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T11:16:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (296 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T11:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:32Z Disabled control of system clock
I20260812 11:16:32.941429 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:34999
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:34999
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/testchainca.cert with env {}
W20260812 11:16:33.046363 10165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:33.046576 10165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:33.046612 10165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:33.050415 10165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:33.050457 10165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:33.050472 10165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:33.050490 10165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:33.050506 10165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:16:33.050525 10165 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:16:33.050537 10165 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 11:16:33.055500 10165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:34999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:34999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:33.056300 10165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:33.057132 10165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:33.064816 10182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:33.064889 10165 server_base.cc:1034] running on GCE node
W20260812 11:16:33.064896 10171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:33.064910 10172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:33.065563 10165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:33.066533 10165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:33.067685 10165 hybrid_clock.cc:648] HybridClock initialized: now 1786533393067665 us; error 30 us; skew 500 ppm
I20260812 11:16:33.069334 10165 webserver.cc:460] Webserver started at http://127.30.255.190:34123/ using document root <none> and password file <none>
I20260812 11:16:33.069842 10165 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:33.069906 10165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:33.070103 10165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:33.072170 10165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "15b12eae9ea74e3f809b7825d4a22a9c"
format_stamp: "Formatted at 2026-08-12 11:16:33 on dist-test-slave-gjrz"
server_key: "34215c6bd11a5677c7ea513e6ef3c7a6"
server_key_iv: "899f9a26287180ba6ecbce563c4daa0e"
server_key_version: "encryptionkey@0"
I20260812 11:16:33.072748 10165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "15b12eae9ea74e3f809b7825d4a22a9c"
format_stamp: "Formatted at 2026-08-12 11:16:33 on dist-test-slave-gjrz"
server_key: "34215c6bd11a5677c7ea513e6ef3c7a6"
server_key_iv: "899f9a26287180ba6ecbce563c4daa0e"
server_key_version: "encryptionkey@0"
I20260812 11:16:33.076215 10165 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:16:33.078387 10194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:33.079530 10165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:33.079648 10165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "15b12eae9ea74e3f809b7825d4a22a9c"
format_stamp: "Formatted at 2026-08-12 11:16:33 on dist-test-slave-gjrz"
server_key: "34215c6bd11a5677c7ea513e6ef3c7a6"
server_key_iv: "899f9a26287180ba6ecbce563c4daa0e"
server_key_version: "encryptionkey@0"
I20260812 11:16:33.079741 10165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:33.090131 10165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:33.090696 10165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:33.090852 10165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:33.097220 10165 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:34999
I20260812 11:16:33.097236 10298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:34999 every 8 connection(s)
I20260812 11:16:33.098060 10165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:33.100736 10299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:33.105792 10299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c: Bootstrap starting.
I20260812 11:16:33.106096 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10165
I20260812 11:16:33.106205 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:33.106431 31742 external_mini_cluster.cc:1428] Setting key 1e0b7641fb307c5dedc07b1444d9ed8c
I20260812 11:16:33.108196 10299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:33.108876 10299 log.cc:824] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:33.110556 10299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c: No bootstrap required, opened a new log
I20260812 11:16:33.113763 10299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } }
I20260812 11:16:33.113961 10299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:33.114033 10299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b12eae9ea74e3f809b7825d4a22a9c, State: Initialized, Role: FOLLOWER
I20260812 11:16:33.114440 10299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } }
I20260812 11:16:33.114569 10299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:33.114634 10299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:33.114730 10299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:33.115033 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:33.107599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.30.255.190:34999)
0812 11:16:33.108213 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:33.108228 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:33.108268 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:16:33.108880 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:33.108897 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:33.109299 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:33.109591 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:33.109991 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.110965 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.111890 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.112363 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.112511 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:33.113267 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:33.113272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:33.113284 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:16:33.113633 (+ 349us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:33.113692 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:16:33.113712 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:33.113797 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:33.113940 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 11:16:33.114312 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":123,"threads_started":1}
I20260812 11:16:33.115589 10299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } }
I20260812 11:16:33.115890 10299 leader_election.cc:302] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15b12eae9ea74e3f809b7825d4a22a9c; no voters:
I20260812 11:16:33.116168 10299 leader_election.cc:288] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:33.116245 10310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:33.116415 10310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [term 1 LEADER]: Becoming Leader. State: Replica: 15b12eae9ea74e3f809b7825d4a22a9c, State: Running, Role: LEADER
I20260812 11:16:33.116753 10310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } }
I20260812 11:16:33.117341 10299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:33.117573 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:16:33.118389 10312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } } }
I20260812 11:16:33.118520 10312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:33.118793 10313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b12eae9ea74e3f809b7825d4a22a9c. Latest consensus state: current_term: 1 leader_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b12eae9ea74e3f809b7825d4a22a9c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 34999 } } }
I20260812 11:16:33.118944 10313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:33.119355 10329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:33.121982 10329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:33.127028 10329 catalog_manager.cc:1347] Generated new cluster ID: 12ec9ad6da2947aabf1133f264d90cb1
I20260812 11:16:33.127089 10329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:33.133495 10329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:33.134558 10329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:33.142833 10329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15b12eae9ea74e3f809b7825d4a22a9c: Generated new TSK 0
I20260812 11:16:33.143512 10329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:16:33.187429 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:33.158571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.30.255.190:34999)
0812 11:16:33.158706 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:33.158711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:33.158884 (+ 173us) server_negotiation.cc:407] Connection header received
0812 11:16:33.159029 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:33.159032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:33.159074 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:33.159163 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:16:33.159421 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.159936 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.161135 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.161344 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.161413 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:33.161721 (+ 308us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:16:33.187079 (+ 25358us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 11:16:33.187328 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43114: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 11:16:33.187491 10307 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43114: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 11:16:33.187649 10354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:33.158497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:34999 (local address 127.0.0.1:43114)
0812 11:16:33.158788 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:33.158803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:33.158931 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:33.159180 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:33.159183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:33.159191 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:16:33.159331 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:33.159338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.160033 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.160036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:33.161029 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:33.161039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:33.161453 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:33.161457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:33.161552 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:33.161588 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:16:33.187292 (+ 25704us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 11:16:33.187465 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.255.190:34999: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":99,"threads_started":1}
I20260812 11:16:33.188227 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10165
2026-08-12T11:16:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (281 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (120 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:33 dist-test-slave-gjrz krb5kdc[10378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:33 dist-test-slave-gjrz krb5kdc[10378](info): set up 2 sockets
Aug 12 11:16:33 dist-test-slave-gjrz krb5kdc[10378](info): commencing operation
krb5kdc: starting...
W20260812 11:16:34.625468 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:34 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:34.755716 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:40843
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:40843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:34.862454 10398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:34.862672 10398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:34.862712 10398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:34.866545 10398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:34.866595 10398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:34.866613 10398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:34.866633 10398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:34.866652 10398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:34.871562 10398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:40843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:40843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:34.872351 10398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:34.873226 10398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:34.880857 10405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:34.880901 10404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:34.880901 10408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:34.880999 10398 server_base.cc:1034] running on GCE node
I20260812 11:16:34.881587 10398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:34.882530 10398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:34.883679 10398 hybrid_clock.cc:648] HybridClock initialized: now 1786533394883662 us; error 34 us; skew 500 ppm
Aug 12 11:16:34 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533394, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:34.886149 10398 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:34.886881 10398 webserver.cc:460] Webserver started at http://127.30.255.190:40585/ using document root <none> and password file <none>
I20260812 11:16:34.887382 10398 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:34.887445 10398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:34.887599 10398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:34.889384 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "445d562e93e4471ea52410847a04a352"
format_stamp: "Formatted at 2026-08-12 11:16:34 on dist-test-slave-gjrz"
server_key: "68eb203f60a553ba1a50fcad4fd9a4e3"
server_key_iv: "3c0680242e2e8c49b0179a2482e26c13"
server_key_version: "encryptionkey@0"
I20260812 11:16:34.889865 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "445d562e93e4471ea52410847a04a352"
format_stamp: "Formatted at 2026-08-12 11:16:34 on dist-test-slave-gjrz"
server_key: "68eb203f60a553ba1a50fcad4fd9a4e3"
server_key_iv: "3c0680242e2e8c49b0179a2482e26c13"
server_key_version: "encryptionkey@0"
I20260812 11:16:34.893214 10398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:34.895355 10425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:34.896448 10398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:34.896543 10398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "445d562e93e4471ea52410847a04a352"
format_stamp: "Formatted at 2026-08-12 11:16:34 on dist-test-slave-gjrz"
server_key: "68eb203f60a553ba1a50fcad4fd9a4e3"
server_key_iv: "3c0680242e2e8c49b0179a2482e26c13"
server_key_version: "encryptionkey@0"
I20260812 11:16:34.896620 10398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:34.909574 10398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:34.912062 10398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:34.912212 10398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:34.918547 10398 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:40843
I20260812 11:16:34.918561 10528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:40843 every 8 connection(s)
I20260812 11:16:34.919459 10398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:34.920708 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10398
I20260812 11:16:34.920835 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:34.921164 31742 external_mini_cluster.cc:1428] Setting key 42c10a154a8f7990307ad68765f38ec9
I20260812 11:16:34.922576 10529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:34.927899 10529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: Bootstrap starting.
Aug 12 11:16:34 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:34.930127 10529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:34.930934 10529 log.cc:824] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:34.933001 10529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: No bootstrap required, opened a new log
I20260812 11:16:34.934543 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:34.922537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.30.255.190:40843)
0812 11:16:34.922973 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:34.922988 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:34.923023 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:16:34.923678 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:34.923696 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:34.924004 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:34.924298 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:34.925099 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:34.926072 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:34.926911 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:34.927417 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:34.927563 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:34.930012 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:34.930018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:34.930030 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:34.930057 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:34.931774 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:34.932254 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:34.932260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:34.932266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:34.932380 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:34.932688 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:34.932690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:34.932691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:34.933041 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:34.933198 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:34.933595 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 11:16:34.933871 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":140,"threads_started":1}
I20260812 11:16:34.936673 10529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } }
I20260812 11:16:34.936903 10529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:34.936962 10529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445d562e93e4471ea52410847a04a352, State: Initialized, Role: FOLLOWER
I20260812 11:16:34.937367 10529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } }
I20260812 11:16:34.937494 10529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:34.937557 10529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:34.937650 10529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:34.938422 10529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } }
I20260812 11:16:34.938722 10529 leader_election.cc:302] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 445d562e93e4471ea52410847a04a352; no voters:
I20260812 11:16:34.938980 10529 leader_election.cc:288] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:34.939095 10542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:34.939332 10542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [term 1 LEADER]: Becoming Leader. State: Replica: 445d562e93e4471ea52410847a04a352, State: Running, Role: LEADER
I20260812 11:16:34.939666 10542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } }
I20260812 11:16:34.940084 10529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:34.941124 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "445d562e93e4471ea52410847a04a352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } } }
I20260812 11:16:34.941254 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:34.941588 10546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 445d562e93e4471ea52410847a04a352. Latest consensus state: current_term: 1 leader_uuid: "445d562e93e4471ea52410847a04a352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445d562e93e4471ea52410847a04a352" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 40843 } } }
I20260812 11:16:34.941766 10546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352 [sys.catalog]: This master's current role is: LEADER
W20260812 11:16:34.944473 10567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:34.944540 10567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:34.944610 10555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:34.945482 10555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:34.950023 10555 catalog_manager.cc:1347] Generated new cluster ID: cb2392b827f64815b96f53813dbdd008
I20260812 11:16:34.950083 10555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:34.963364 10555 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:34.964198 10555 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:34.970255 10555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 445d562e93e4471ea52410847a04a352: Generated new TSK 0
I20260812 11:16:34.970901 10555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:35.001981 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:35.109474 10583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:35.109684 10583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:35.109740 10583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:35.113809 10583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:35.113900 10583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:35.113978 10583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:35.119220 10583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:35.120150 10583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:35.121065 10583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:35.129844 10591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.129854 10592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.129854 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:35.130263 10583 server_base.cc:1034] running on GCE node
I20260812 11:16:35.130614 10583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:35.131171 10583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:35.132334 10583 hybrid_clock.cc:648] HybridClock initialized: now 1786533395132310 us; error 39 us; skew 500 ppm
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.134951 10583 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:35.135744 10583 webserver.cc:460] Webserver started at http://127.30.255.129:36119/ using document root <none> and password file <none>
I20260812 11:16:35.136235 10583 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:35.136300 10583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:35.136473 10583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:35.138336 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "cb6cac462ad74233b5b6b0684e78f38f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "34af1d5146b24d162adaf237da5522f5"
server_key_iv: "6cf6ad08fb5513f205ccd4941dc79418"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.138854 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "cb6cac462ad74233b5b6b0684e78f38f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "34af1d5146b24d162adaf237da5522f5"
server_key_iv: "6cf6ad08fb5513f205ccd4941dc79418"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.142400 10583 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:16:35.144589 10615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.145664 10583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:35.145768 10583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "cb6cac462ad74233b5b6b0684e78f38f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "34af1d5146b24d162adaf237da5522f5"
server_key_iv: "6cf6ad08fb5513f205ccd4941dc79418"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.145852 10583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:35.165674 10583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:35.168145 10583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:35.168299 10583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:35.168802 10583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:35.169548 10583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:35.169598 10583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.169647 10583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:35.169675 10583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.177891 10583 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:43433
I20260812 11:16:35.177922 10854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:43433 every 8 connection(s)
I20260812 11:16:35.178668 10583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:35.187652 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10583
I20260812 11:16:35.187786 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:35.188109 31742 external_mini_cluster.cc:1428] Setting key 1e85377b6c98673c00f0d81df07f08df
I20260812 11:16:35.191785 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.180337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:51893 (local address 127.30.255.190:40843)
0812 11:16:35.180457 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.180461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.181210 (+ 749us) server_negotiation.cc:407] Connection header received
0812 11:16:35.182510 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.182513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.182570 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.182645 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.183805 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.184366 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.185539 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.185776 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.185864 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.188264 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.188268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.188270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.188295 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.189734 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.190274 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.190277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.190278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.190348 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.190683 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.190685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.190686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.190846 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.190914 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.191504 (+ 590us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.191629 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:35.192483 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.180650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40843 (local address 127.30.255.129:51893)
0812 11:16:35.181063 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:35.181103 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:35.182309 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:35.182832 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:35.182839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:35.183163 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:35.183656 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.183671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.184524 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.184527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.185411 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.185422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.185914 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.185918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.186074 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.186567 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:35.186582 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:35.188073 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:35.189861 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.189867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.189880 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.190186 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.190421 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.190424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.190425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.190602 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.190930 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:35.190936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:35.191212 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 11:16:35.191453 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 11:16:35.191715 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":103,"threads_started":1}
I20260812 11:16:35.193576 10861 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40843
I20260812 11:16:35.193806 10861 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:35.194379 10861 heartbeater.cc:511] Master 127.30.255.190:40843 requested a full tablet report, sending...
I20260812 11:16:35.195855 10458 ts_manager.cc:194] Registered new tserver with Master: cb6cac462ad74233b5b6b0684e78f38f (127.30.255.129:43433)
I20260812 11:16:35.196920 10458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:51893
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:35.214344 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:35.319831 10869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:35.320061 10869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:35.320118 10869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:35.324021 10869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:35.324074 10869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:35.324150 10869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:35.329171 10869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:35.330098 10869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:35.331053 10869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:35.339988 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.340025 10878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.339999 10876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:35.340286 10869 server_base.cc:1034] running on GCE node
I20260812 11:16:35.340801 10869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:35.341434 10869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:35.342599 10869 hybrid_clock.cc:648] HybridClock initialized: now 1786533395342576 us; error 26 us; skew 500 ppm
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.345160 10869 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:35.345986 10869 webserver.cc:460] Webserver started at http://127.30.255.130:41319/ using document root <none> and password file <none>
I20260812 11:16:35.346493 10869 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:35.346558 10869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:35.346755 10869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:35.348769 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "6acd107aca9246e09a481272b98c00b7"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "22f1e7bf80308bff12929bb418064f83"
server_key_iv: "ff530d2e8358cd246f1d4e216bbdfc64"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.349298 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "6acd107aca9246e09a481272b98c00b7"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "22f1e7bf80308bff12929bb418064f83"
server_key_iv: "ff530d2e8358cd246f1d4e216bbdfc64"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.352828 10869 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:35.355024 10901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.356177 10869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:35.356276 10869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "6acd107aca9246e09a481272b98c00b7"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "22f1e7bf80308bff12929bb418064f83"
server_key_iv: "ff530d2e8358cd246f1d4e216bbdfc64"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.356369 10869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:35.368044 10869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:35.370478 10869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:35.370642 10869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:35.371160 10869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:35.371942 10869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:35.371991 10869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.372027 10869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:35.372043 10869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.380231 10869 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:39273
I20260812 11:16:35.380260 11120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:39273 every 8 connection(s)
I20260812 11:16:35.381028 10869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:35.389343 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10869
I20260812 11:16:35.389480 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:35.389784 31742 external_mini_cluster.cc:1428] Setting key 08dbcd95aa1aa1d538b8b19e322c65a9
I20260812 11:16:35.393519 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.382712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:47303 (local address 127.30.255.190:40843)
0812 11:16:35.382837 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.382841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.383641 (+ 800us) server_negotiation.cc:407] Connection header received
0812 11:16:35.384894 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.384897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.384943 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.385012 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.386034 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.386540 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.387636 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.387843 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.387910 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.390220 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.390224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.390226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.390250 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.391647 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.392111 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.392114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.392116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.392206 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.392515 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.392519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.392521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.392696 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.392787 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.393248 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.393365 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:35.394232 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.383034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40843 (local address 127.30.255.130:47303)
0812 11:16:35.383493 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:35.383533 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:35.384713 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:35.385156 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:35.385163 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:35.385476 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:35.385924 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.385938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.386627 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.386630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.387538 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.387548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.387942 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.387944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.388084 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.388526 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:35.388541 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:35.390057 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:35.391755 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.391761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.391774 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.392016 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.392290 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.392293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.392295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.392428 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.392803 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:35.392809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:35.393025 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 11:16:35.393186 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:16:35.393472 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":110,"threads_started":1}
I20260812 11:16:35.395331 11121 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40843
I20260812 11:16:35.395545 11121 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:35.396008 11121 heartbeater.cc:511] Master 127.30.255.190:40843 requested a full tablet report, sending...
I20260812 11:16:35.397042 10458 ts_manager.cc:194] Registered new tserver with Master: 6acd107aca9246e09a481272b98c00b7 (127.30.255.130:39273)
I20260812 11:16:35.397783 10458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:47303
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:35.415637 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:35.527822 11130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:35.528026 11130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:35.528070 11130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:35.531973 11130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:35.532021 11130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:35.532089 11130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:35.537230 11130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:35.538096 11130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:35.538995 11130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:35.548058 11141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.548194 11137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:35.548156 11138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:35.548231 11130 server_base.cc:1034] running on GCE node
I20260812 11:16:35.548749 11130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:35.549304 11130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:35.550460 11130 hybrid_clock.cc:648] HybridClock initialized: now 1786533395550447 us; error 24 us; skew 500 ppm
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.552985 11130 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:35.553771 11130 webserver.cc:460] Webserver started at http://127.30.255.131:40639/ using document root <none> and password file <none>
I20260812 11:16:35.554239 11130 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:35.554291 11130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:35.554448 11130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:35.556315 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "389c91ccaa284aa2afdf8642bd03d39f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "dfe285b04359583afbe3c944893b8fa6"
server_key_iv: "ed776889b0f476a1294508ed19a8d9f4"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.556798 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "389c91ccaa284aa2afdf8642bd03d39f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "dfe285b04359583afbe3c944893b8fa6"
server_key_iv: "ed776889b0f476a1294508ed19a8d9f4"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.560228 11130 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:35.562392 11149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.563516 11130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:35.563628 11130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "389c91ccaa284aa2afdf8642bd03d39f"
format_stamp: "Formatted at 2026-08-12 11:16:35 on dist-test-slave-gjrz"
server_key: "dfe285b04359583afbe3c944893b8fa6"
server_key_iv: "ed776889b0f476a1294508ed19a8d9f4"
server_key_version: "encryptionkey@0"
I20260812 11:16:35.563715 11130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:35.578047 11130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:35.580422 11130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:35.580581 11130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:35.581074 11130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:35.581804 11130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:35.581851 11130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.581899 11130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:35.581929 11130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:35.589978 11130 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:35085
I20260812 11:16:35.590026 11359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:35085 every 8 connection(s)
I20260812 11:16:35.590754 11130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:35.600765 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11130
I20260812 11:16:35.600881 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:35.601215 31742 external_mini_cluster.cc:1428] Setting key f5c8af9a69737210d1c9e36ea311a58c
I20260812 11:16:35.602839 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.592362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:53107 (local address 127.30.255.190:40843)
0812 11:16:35.592493 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.592498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.593277 (+ 779us) server_negotiation.cc:407] Connection header received
0812 11:16:35.594477 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.594480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.594526 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.594600 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.595650 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.596162 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.597204 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.597402 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.597465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.599594 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.599597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.599599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.599624 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.600730 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.601272 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.601275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.601277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.601366 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.601701 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.601704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.601705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.601909 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.601998 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.602570 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.602688 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:35.603582 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.592707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:40843 (local address 127.30.255.131:53107)
0812 11:16:35.593143 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:35.593183 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:35.594302 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:35.594755 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:35.594761 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:35.595073 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:35.595540 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.595553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.596251 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.596253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.597109 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:35.597119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.597499 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.597502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:35.597639 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.598093 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:35.598107 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:35.599429 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:35.600838 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.600845 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.600859 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.601178 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.601451 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:35.601454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:35.601456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:35.601602 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:35.602026 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:35.602032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:35.602314 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 11:16:35.602505 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 11:16:35.602772 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":109,"threads_started":1}
I20260812 11:16:35.604537 11362 heartbeater.cc:347] Connected to a master server at 127.30.255.190:40843
I20260812 11:16:35.604748 11362 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:35.605172 11362 heartbeater.cc:511] Master 127.30.255.190:40843 requested a full tablet report, sending...
I20260812 11:16:35.606055 10458 ts_manager.cc:194] Registered new tserver with Master: 389c91ccaa284aa2afdf8642bd03d39f (127.30.255.131:35085)
I20260812 11:16:35.606566 10458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:53107
I20260812 11:16:35.614643 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/test-admin.keytab.
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.631351 31742 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 11:16:35.634109 11374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:35.632632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.30.255.129:43433)
0812 11:16:35.632966 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.632976 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.632996 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:16:35.633042 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.633047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.633275 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.633455 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.633934 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42322: BlockingRecv error: recv got EOF from 127.0.0.1:42322 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":104,"threads_started":1}
W20260812 11:16:35.634177 31742 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.635149 31742 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.635999 31742 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:35.644305 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.636738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.30.255.129:43433)
0812 11:16:35.636863 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.636866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.636924 (+ 58us) server_negotiation.cc:407] Connection header received
0812 11:16:35.637026 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.637028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.637063 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.637116 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.637549 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.638317 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.639049 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.639431 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.639509 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.641489 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.641501 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.641517 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.641540 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.642816 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.643184 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.643188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.643189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.643294 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.643510 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.643512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.643513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.643743 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.643895 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.644019 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.644172 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 11:16:35.647604 11374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:35.646758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.30.255.129:43433)
0812 11:16:35.646878 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.646881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.646918 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:16:35.647010 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.647012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.647044 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.647101 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.647521 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42342: BlockingRecv error: recv got EOF from 127.0.0.1:42342 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 11:16:35.647722 31742 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.648594 31742 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.649418 31742 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:35.656394 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.650118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.30.255.129:43433)
0812 11:16:35.650227 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.650230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.650317 (+ 87us) server_negotiation.cc:407] Connection header received
0812 11:16:35.650398 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.650401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.650432 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.650489 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.650862 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.651383 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.652064 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.652272 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.652335 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.654151 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.654154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.654155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.654176 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.655208 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.655558 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.655560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.655561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.655628 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.655814 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.655816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.655817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.655946 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.656027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.656181 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.656265 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 11:16:35.658429 11374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:35.657731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.30.255.129:43433)
0812 11:16:35.657848 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.657852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.657906 (+ 54us) server_negotiation.cc:407] Connection header received
0812 11:16:35.657997 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.657999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.658032 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.658081 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.658354 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42362: BlockingRecv error: recv got EOF from 127.0.0.1:42362 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 11:16:35.658551 31742 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786533365564394-31742-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.659497 31742 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:35.660344 31742 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[10378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:16:35.667372 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:35.661024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.30.255.129:43433)
0812 11:16:35.661152 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:35.661155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:35.661169 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:35.661261 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:35.661263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:35.661293 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:35.661349 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:35.661721 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.662173 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.662835 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:35.663017 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:35.663078 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:35.664922 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:35.664926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:35.664927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:35.664947 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:35.666097 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.666398 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.666400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.666401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.666509 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:35.666689 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:35.666691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:35.666692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:35.666861 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:35.666965 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:35.667150 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 11:16:35.667238 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:16:35.668154 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10583
I20260812 11:16:35.683722 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10869
I20260812 11:16:35.692101 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11130
I20260812 11:16:35.703626 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10398
2026-08-12T11:16:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2274 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[11398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[11398](info): set up 2 sockets
Aug 12 11:16:35 dist-test-slave-gjrz krb5kdc[11398](info): commencing operation
krb5kdc: starting...
W20260812 11:16:36.904261 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:36 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab.
Entry for principal oryx/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab.
I20260812 11:16:37.023586 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38763
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab
--principal=oryx/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:37.131934 11422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:37.132164 11422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:37.132200 11422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:37.136086 11422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:37.136138 11422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:37.136157 11422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:37.136170 11422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:37.136188 11422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:37.141155 11422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab
--principal=oryx/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:38763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:37.142035 11422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:37.142918 11422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:37.150822 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.150796 11430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.150874 11422 server_base.cc:1034] running on GCE node
W20260812 11:16:37.150837 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.151561 11422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:37.152544 11422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:37.153700 11422 hybrid_clock.cc:648] HybridClock initialized: now 1786533397153683 us; error 29 us; skew 500 ppm
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:37.156311 11422 init.cc:377] Logged in from keytab as oryx/127.30.255.190@KRBTEST.COM (short username oryx)
I20260812 11:16:37.157124 11422 webserver.cc:460] Webserver started at http://127.30.255.190:39829/ using document root <none> and password file <none>
I20260812 11:16:37.157600 11422 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:37.157653 11422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:37.157811 11422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:37.159915 11422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "7994d6ab0efa484886eb818b2eb0008d"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "e8608baa448e68a0679976e775988abe"
server_key_iv: "fc7ee4a2ee19a305eee95840c7685560"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.160435 11422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "7994d6ab0efa484886eb818b2eb0008d"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "e8608baa448e68a0679976e775988abe"
server_key_iv: "fc7ee4a2ee19a305eee95840c7685560"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.164008 11422 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:37.166234 11450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.167397 11422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:37.167495 11422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "7994d6ab0efa484886eb818b2eb0008d"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "e8608baa448e68a0679976e775988abe"
server_key_iv: "fc7ee4a2ee19a305eee95840c7685560"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.167577 11422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:37.194186 11422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:37.197151 11422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:37.197300 11422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:37.203881 11422 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:38763
I20260812 11:16:37.204144 11554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:38763 every 8 connection(s)
I20260812 11:16:37.204799 11422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:37.207497 11555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:37.208981 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11422
I20260812 11:16:37.209111 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:37.209432 31742 external_mini_cluster.cc:1428] Setting key c24aa1806ea4428a4db35ccd5fb2a094
I20260812 11:16:37.213583 11555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Bootstrap starting.
I20260812 11:16:37.215977 11555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:37.216832 11555 log.cc:824] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:37.218940 11555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: No bootstrap required, opened a new log
I20260812 11:16:37.223255 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.210709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.30.255.190:38763)
0812 11:16:37.211093 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:37.211109 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:37.211146 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:16:37.211808 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:37.211825 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:37.212134 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:37.212397 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:37.213255 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.214229 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.215371 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.215833 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.215945 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.218433 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:37.218440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:37.218455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:37.218485 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:37.220340 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.220750 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.220756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.220761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.220912 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.221148 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.221151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.221153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.221640 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:37.221827 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:37.222020 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 11:16:37.222317 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260812 11:16:37.223281 11555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } }
I20260812 11:16:37.223605 11555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:37.223675 11555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7994d6ab0efa484886eb818b2eb0008d, State: Initialized, Role: FOLLOWER
I20260812 11:16:37.224107 11555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } }
I20260812 11:16:37.224267 11555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:37.224406 11555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:37.224511 11555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:37.225340 11555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } }
I20260812 11:16:37.225684 11555 leader_election.cc:302] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7994d6ab0efa484886eb818b2eb0008d; no voters:
I20260812 11:16:37.225976 11555 leader_election.cc:288] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:37.226069 11563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:37.226271 11563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 LEADER]: Becoming Leader. State: Replica: 7994d6ab0efa484886eb818b2eb0008d, State: Running, Role: LEADER
I20260812 11:16:37.226580 11563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } }
I20260812 11:16:37.227025 11555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:37.228096 11564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } }
I20260812 11:16:37.228247 11564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:37.228252 11565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7994d6ab0efa484886eb818b2eb0008d. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } }
I20260812 11:16:37.228551 11565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:37.228845 11590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:37.231492 11590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:37.236616 11590 catalog_manager.cc:1347] Generated new cluster ID: a1fafdfbec5649a498ce79bff06324e7
I20260812 11:16:37.236677 11590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:37.242257 11590 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:37.243428 11590 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:37.251824 11590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Generated new TSK 0
I20260812 11:16:37.252398 11590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
I20260812 11:16:37.282127 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:37.390655 11611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:37.390869 11611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:37.390916 11611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:37.394752 11611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:37.394799 11611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:37.394852 11611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:37.399761 11611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:37.400621 11611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:37.401494 11611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:37.410265 11621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.410274 11625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.410714 11611 server_base.cc:1034] running on GCE node
W20260812 11:16:37.410791 11623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.411140 11611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:37.411755 11611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:37.412951 11611 hybrid_clock.cc:648] HybridClock initialized: now 1786533397412933 us; error 28 us; skew 500 ppm
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:37.415475 11611 init.cc:377] Logged in from keytab as oryx/127.30.255.129@KRBTEST.COM (short username oryx)
I20260812 11:16:37.416213 11611 webserver.cc:460] Webserver started at http://127.30.255.129:42501/ using document root <none> and password file <none>
I20260812 11:16:37.416685 11611 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:37.416746 11611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:37.416930 11611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:37.418756 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "84717fdae41747658340f4b452e62248"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "26b5f78d654ae4a0efba1adeed16aa5c"
server_key_iv: "2399f4e588ba36973f04526904e24aa0"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.419251 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "84717fdae41747658340f4b452e62248"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "26b5f78d654ae4a0efba1adeed16aa5c"
server_key_iv: "2399f4e588ba36973f04526904e24aa0"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.422704 11611 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260812 11:16:37.424965 11641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.426096 11611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:37.426200 11611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "84717fdae41747658340f4b452e62248"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "26b5f78d654ae4a0efba1adeed16aa5c"
server_key_iv: "2399f4e588ba36973f04526904e24aa0"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.426290 11611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:37.443521 11611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:37.445875 11611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:37.446038 11611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:37.446532 11611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:37.447291 11611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:37.447355 11611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.447405 11611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:37.447431 11611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.455506 11611 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:36147
I20260812 11:16:37.455534 11858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:36147 every 8 connection(s)
I20260812 11:16:37.456290 11611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:37.457476 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11611
I20260812 11:16:37.457592 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:37.457927 31742 external_mini_cluster.cc:1428] Setting key 0c9fdda74f60ce8ac59030f4c73c8076
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:37.470561 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.457976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:39115 (local address 127.30.255.190:38763)
0812 11:16:37.458099 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:37.458103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:37.458985 (+ 882us) server_negotiation.cc:407] Connection header received
0812 11:16:37.460365 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:37.460368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:37.460424 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:37.460497 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:37.461844 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.462522 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.464075 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.464307 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.464389 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.467224 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:37.467227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:37.467229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:37.467252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:37.468545 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.469049 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.469052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.469053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.469137 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.469495 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.469498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.469500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.469688 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:37.469779 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:37.470275 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 11:16:37.470398 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:37.471570 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.458416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:39115)
0812 11:16:37.458851 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:37.458891 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:37.460163 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:37.460680 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:37.460687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:37.461027 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:37.461688 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.461709 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.462665 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.462668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.463953 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.463966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.464469 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.464474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.464780 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.465343 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:37.465365 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:37.467056 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:37.468661 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.468667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.468680 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.468935 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.469217 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.469219 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.469220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.469401 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.469792 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:37.469799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:37.470019 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 11:16:37.470216 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 11:16:37.470639 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260812 11:16:37.473664 11861 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:37.473868 11861 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:37.474283 11861 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
I20260812 11:16:37.475682 11481 ts_manager.cc:194] Registered new tserver with Master: 84717fdae41747658340f4b452e62248 (127.30.255.129:36147)
I20260812 11:16:37.476786 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:39115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
I20260812 11:16:37.485857 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:37.594059 11873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:37.594283 11873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:37.594341 11873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:37.598246 11873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:37.598299 11873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:37.598376 11873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:37.603281 11873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:37.604177 11873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:37.605072 11873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:37.613878 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.613871 11883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.613907 11888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.614048 11873 server_base.cc:1034] running on GCE node
I20260812 11:16:37.614487 11873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:37.615001 11873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:37.616186 11873 hybrid_clock.cc:648] HybridClock initialized: now 1786533397616150 us; error 45 us; skew 500 ppm
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:37.618721 11873 init.cc:377] Logged in from keytab as oryx/127.30.255.130@KRBTEST.COM (short username oryx)
I20260812 11:16:37.619563 11873 webserver.cc:460] Webserver started at http://127.30.255.130:41859/ using document root <none> and password file <none>
I20260812 11:16:37.620066 11873 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:37.620131 11873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:37.620311 11873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:37.622169 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "83f4c764308448059050d97df4271080"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "d7ce0d65f2cccaa012d39b581161c1a2"
server_key_iv: "bf032f16c42a65d5a777480e70381191"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.622694 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "83f4c764308448059050d97df4271080"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "d7ce0d65f2cccaa012d39b581161c1a2"
server_key_iv: "bf032f16c42a65d5a777480e70381191"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.626205 11873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:16:37.628769 11901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.629921 11873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:37.630028 11873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "83f4c764308448059050d97df4271080"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "d7ce0d65f2cccaa012d39b581161c1a2"
server_key_iv: "bf032f16c42a65d5a777480e70381191"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.630132 11873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:37.652802 11873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:37.655376 11873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:37.655544 11873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:37.656091 11873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:37.656876 11873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:37.656924 11873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.656975 11873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:37.657003 11873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.665215 11873 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:35291
I20260812 11:16:37.665247 12108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:35291 every 8 connection(s)
I20260812 11:16:37.666015 11873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:37.671347 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11873
I20260812 11:16:37.671455 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:37.671721 31742 external_mini_cluster.cc:1428] Setting key fde4274fd8e6e08a38f9b1723b4beb88
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:37.679493 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.667683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:47319 (local address 127.30.255.190:38763)
0812 11:16:37.667814 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:37.667818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:37.668574 (+ 756us) server_negotiation.cc:407] Connection header received
0812 11:16:37.669888 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:37.669891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:37.669949 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:37.670021 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:37.671162 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.671855 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.673202 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.673415 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.673494 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.676084 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:37.676089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:37.676091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:37.676120 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:37.677419 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.677961 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.677963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.677964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.678035 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.678400 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.678403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.678404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.678599 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:37.678684 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:37.679240 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 11:16:37.679377 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:37.680541 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.668022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:47319)
0812 11:16:37.668435 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:37.668475 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:37.669683 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:37.670203 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:37.670210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:37.670540 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:37.671017 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.671031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.671985 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.671988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.673071 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.673085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.673567 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.673572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.673767 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.674284 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:37.674304 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:37.675923 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:37.677535 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.677541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.677562 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.677877 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.678123 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.678127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.678129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.678315 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.678706 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:37.678712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:37.679008 (+ 296us) client_negotiation.cc:769] Sending connection context
0812 11:16:37.679218 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 11:16:37.679572 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260812 11:16:37.681798 12110 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:37.682017 12110 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:37.682466 12110 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:37.683482 11481 ts_manager.cc:194] Registered new tserver with Master: 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:37.684023 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:47319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
I20260812 11:16:37.699920 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:37.809083 12122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:37.809297 12122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:37.809342 12122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:37.813198 12122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:37.813246 12122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:37.813309 12122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:37.818284 12122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:37.819141 12122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:37.820034 12122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:37.828907 12133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.828940 12131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:37.828940 12128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:37.829070 12122 server_base.cc:1034] running on GCE node
I20260812 11:16:37.829519 12122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:37.830068 12122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:37.831224 12122 hybrid_clock.cc:648] HybridClock initialized: now 1786533397831207 us; error 27 us; skew 500 ppm
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:37.833736 12122 init.cc:377] Logged in from keytab as oryx/127.30.255.131@KRBTEST.COM (short username oryx)
I20260812 11:16:37.834538 12122 webserver.cc:460] Webserver started at http://127.30.255.131:43707/ using document root <none> and password file <none>
I20260812 11:16:37.835074 12122 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:37.835136 12122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:37.835371 12122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:37.837282 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "fd06182d8bc84772926e4aecaa712379"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "6dc9597deaa805eb9f7fe1b3c75d778f"
server_key_iv: "73a1f197edec6e6a38591f4a53c041c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.837796 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "fd06182d8bc84772926e4aecaa712379"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "6dc9597deaa805eb9f7fe1b3c75d778f"
server_key_iv: "73a1f197edec6e6a38591f4a53c041c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.841228 12122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:37.843416 12144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.844524 12122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:37.844628 12122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "fd06182d8bc84772926e4aecaa712379"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "6dc9597deaa805eb9f7fe1b3c75d778f"
server_key_iv: "73a1f197edec6e6a38591f4a53c041c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:37.844720 12122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:37.865906 12122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:37.868403 12122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:37.868556 12122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:37.869064 12122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:37.869818 12122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:37.869865 12122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.869905 12122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:37.869933 12122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:37.878429 12122 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:39277
I20260812 11:16:37.878459 12352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:39277 every 8 connection(s)
I20260812 11:16:37.879201 12122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:37.885670 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 12122
I20260812 11:16:37.885787 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:37.886077 31742 external_mini_cluster.cc:1428] Setting key 47e37357c0822fc1b555cb99ed775da5
Aug 12 11:16:37 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:37.890959 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.880818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:49061 (local address 127.30.255.190:38763)
0812 11:16:37.880958 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:37.880963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:37.881741 (+ 778us) server_negotiation.cc:407] Connection header received
0812 11:16:37.882963 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:37.882965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:37.883012 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:37.883082 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:37.884105 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.884625 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.885698 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.886014 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.886125 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.888317 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:37.888320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:37.888322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:37.888346 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:37.889403 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.889837 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.889840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.889841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.889910 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:37.890148 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:37.890150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:37.890151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:37.890304 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:37.890374 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:37.890759 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 11:16:37.890845 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:16:37.891680 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:37.881181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:49061)
0812 11:16:37.881598 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:37.881638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:37.882779 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:37.883228 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:37.883235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:37.883561 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:37.883989 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.884002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.884718 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.884720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.885574 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:37.885583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:37.886160 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:37.886164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:37.886315 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:37.886801 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:37.886816 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:37.888178 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:37.889506 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.889511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.889521 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.889764 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.889971 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:37.889973 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:37.889974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:37.890086 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:37.890372 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:37.890376 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:37.890584 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 11:16:37.890726 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 11:16:37.890938 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":114,"threads_started":1}
I20260812 11:16:37.892653 12355 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:37.892832 12355 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:37.893227 12355 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
I20260812 11:16:37.894162 11481 ts_manager.cc:194] Registered new tserver with Master: fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:37.894693 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:49061
I20260812 11:16:37.899438 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.255.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.189.keytab.
Entry for principal oryx/127.30.255.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.189.keytab.
I20260812 11:16:37.923614 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:38361
--webserver_interface=127.30.255.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.189.keytab
--principal=oryx/127.30.255.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:38763,127.30.255.189:38361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:38.029511 12362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:38.029723 12362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:38.029776 12362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:38.033663 12362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:38.033712 12362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:38.033737 12362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:38.033775 12362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:38.033807 12362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:38.038813 12362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:38763,127.30.255.189:38361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.189.keytab
--principal=oryx/127.30.255.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:38361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.255.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
W20260812 11:16:38.039001 12362 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.255.190:38763,127.30.255.189:38361'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 11:16:38.039742 12362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:38.040599 12362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.048431 12369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.048439 12374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.048499 12362 server_base.cc:1034] running on GCE node
W20260812 11:16:38.048449 12370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.049151 12362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.050153 12362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.051334 12362 hybrid_clock.cc:648] HybridClock initialized: now 1786533398051319 us; error 30 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.053876 12362 init.cc:377] Logged in from keytab as oryx/127.30.255.189@KRBTEST.COM (short username oryx)
I20260812 11:16:38.054659 12362 webserver.cc:460] Webserver started at http://127.30.255.189:43087/ using document root <none> and password file <none>
I20260812 11:16:38.055131 12362 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.055186 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.055384 12362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:38.057264 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/instance:
uuid: "353be5983f544708a027d37acbe885b4"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "3996ad2d23a9bb99a500a6182f72d200"
server_key_iv: "a6c4869be9c0bdd637ff4be51d87ab83"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.057727 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal/instance:
uuid: "353be5983f544708a027d37acbe885b4"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "3996ad2d23a9bb99a500a6182f72d200"
server_key_iv: "a6c4869be9c0bdd637ff4be51d87ab83"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.061143 12362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:38.063282 12386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.064404 12362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:38.064509 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "353be5983f544708a027d37acbe885b4"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "3996ad2d23a9bb99a500a6182f72d200"
server_key_iv: "a6c4869be9c0bdd637ff4be51d87ab83"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.064584 12362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.110183 12362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.113044 12362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:38.113183 12362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.119494 12362 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.189:38361
I20260812 11:16:38.119751 12485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.189:38361 every 8 connection(s)
I20260812 11:16:38.120383 12362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
I20260812 11:16:38.123070 12486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:38.126713 12486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:16:38.128791 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 12362
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.189@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.140241 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.128654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.30.255.190:38763)
0812 11:16:38.128806 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.128811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.129641 (+ 830us) server_negotiation.cc:407] Connection header received
0812 11:16:38.130756 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.130760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.130808 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.130908 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.131915 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.132724 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.134065 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.134415 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.134521 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.136900 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.136904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.136907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.136930 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.138222 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.138714 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.138718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.138720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.138797 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.139105 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.139109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.139110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.139383 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.139474 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.139958 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.140082 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:16:38.140931 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.129620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.30.255.189:38361)
0812 11:16:38.129998 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.130009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.130200 (+ 191us) server_negotiation.cc:407] Connection header received
0812 11:16:38.130569 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.130583 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.130785 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.130954 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.131805 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.133026 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.134146 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.134695 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.134769 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.137482 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.137488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.137505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.137526 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.138912 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.139327 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.139332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.139333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.139408 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.139682 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.139684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.139686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.139915 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.140084 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.140630 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.140745 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"spinlock_wait_cycles":1280,"thread_start_us":129,"threads_started":1}
I20260812 11:16:38.140936 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.129053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51636)
0812 11:16:38.129498 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.129548 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.130569 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.130959 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.130965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.131442 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.131779 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.131800 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.132835 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.132837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.133946 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.133957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.134551 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.134554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.134696 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.135190 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.135211 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.136761 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.138326 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.138335 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.138344 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.138617 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.138881 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.138884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.138886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.139016 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.139484 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.139491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.139735 (+ 244us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.139897 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.140186 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260812 11:16:38.142282 12486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.189@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.148711 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.142706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46350)
0812 11:16:38.142804 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.142818 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.142888 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.143052 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.143054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.143170 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.143251 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.143255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.143905 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.143907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.144394 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.144400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.144795 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.144798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.144872 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.145191 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.145201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.146232 (+ 1031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.147507 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.147510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.147511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.147681 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.147875 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.147877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.147878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.147939 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.148264 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.148267 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.148339 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.148422 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.148555 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260812 11:16:38.148728 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.142821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.30.255.189:38361)
0812 11:16:38.142901 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.142904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.142916 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:38.142945 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.142947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.142976 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.143034 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.143353 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.143817 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.144468 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.144720 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.144789 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.146357 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.146359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.146361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.146382 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.147416 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.147751 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.147754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.147755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.147816 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.147996 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.147998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.147999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.148164 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.148266 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.148486 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.148607 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":22}
I20260812 11:16:38.151717 12486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Bootstrap starting.
I20260812 11:16:38.153841 12486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:38.154488 12486 log.cc:824] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:38.156199 12486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: No bootstrap required, opened a new log
I20260812 11:16:38.158952 12486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } }
I20260812 11:16:38.159129 12486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:38.159168 12486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353be5983f544708a027d37acbe885b4, State: Initialized, Role: FOLLOWER
I20260812 11:16:38.159561 12486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } }
I20260812 11:16:38.159963 12497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } } }
I20260812 11:16:38.160076 12497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:38.160604 12486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 11:16:38.164860 12521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:38.164942 12521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:38.167032 12362 master_runner.cc:428] Detected that this master 353be5983f544708a027d37acbe885b4 is joining an existing cluster
I20260812 11:16:38.167084 12362 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.255.189:38361
I20260812 11:16:38.176859 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.169076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.30.255.190:38763)
0812 11:16:38.169197 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.169202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.169470 (+ 268us) server_negotiation.cc:407] Connection header received
0812 11:16:38.169618 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.169621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.169666 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.169708 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.170432 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.171185 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172072 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.172293 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172385 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.173689 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.173692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.173695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.173720 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.175226 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175762 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.175766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.175768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.175854 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.176104 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.176107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.176109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.176282 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.176367 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.176626 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.176723 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:16:38.176941 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.169096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.30.255.189:38361)
0812 11:16:38.169323 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.169327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.169503 (+ 176us) server_negotiation.cc:407] Connection header received
0812 11:16:38.169614 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.169617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.169664 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.169748 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.170363 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.171115 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172342 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.172659 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172753 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.173917 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.173920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.173923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.173946 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.175406 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175801 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.175804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.175805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.175881 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.176133 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.176135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.176137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.176316 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.176385 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.176680 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.176783 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":167,"spinlock_wait_cycles":4480}
I20260812 11:16:38.176877 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.169000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51646)
0812 11:16:38.169342 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.169355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.169507 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.169837 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.169840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.170048 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.170287 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.170295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.171334 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.171337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.171965 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.171975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.172506 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.172621 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.173147 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.173164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.173563 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.175358 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.175363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.175660 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.175936 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.175941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.176019 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.176373 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.176376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.176473 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.176566 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.176694 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":106,"spinlock_wait_cycles":6528,"thread_start_us":100,"threads_started":1}
I20260812 11:16:38.177443 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.169218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46354)
0812 11:16:38.169443 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.169454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.169541 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.169776 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.169778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.170042 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.170260 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.170268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.171244 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.171248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.172232 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.172243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.172770 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.172774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.172867 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.173282 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.173297 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.173782 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.175493 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.175498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.175725 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.175970 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.175972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.175974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.176045 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.176440 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.176443 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.176515 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.176897 (+ 382us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.177278 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":40,"spinlock_wait_cycles":896,"thread_start_us":95,"threads_started":1}
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.190@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.189347 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.181133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.30.255.189:38361)
0812 11:16:38.181251 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.181255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.181563 (+ 308us) server_negotiation.cc:407] Connection header received
0812 11:16:38.181731 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.181733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.181780 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.181841 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.182536 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.182974 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.184171 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.184358 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.184426 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.186511 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.186514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.186516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.186535 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.187802 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.188233 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.188236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.188237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.188305 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.188549 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.188552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.188554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.188709 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.188791 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.189112 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.189211 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:16:38.189368 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.181088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46360)
0812 11:16:38.181429 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.181484 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.181612 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.181894 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.181900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.182318 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.182439 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.182446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.183062 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.183066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.184052 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.184063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.184449 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.184453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.184559 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.184976 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.184990 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.186385 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.187902 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.187912 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.187917 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.188141 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.188388 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.188391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.188393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.188468 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.188812 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.188818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.188921 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.189039 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.189216 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260812 11:16:38.190501 11481 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.255.189:38361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "353be5983f544708a027d37acbe885b4"
member_type: NON_VOTER
last_known_addr {
host: "127.30.255.189"
port: 38361
}
attrs {
promote: true
}
}
dest_uuid: "7994d6ab0efa484886eb818b2eb0008d"
cas_config_opid_index: -1
I20260812 11:16:38.191042 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } }
I20260812 11:16:38.192502 12549 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 353be5983f544708a027d37acbe885b4 (127.30.255.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } }
I20260812 11:16:38.193260 12442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:38.194253 12549 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.255.189:38361
I20260812 11:16:38.194484 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } }
I20260812 11:16:38.194586 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:38.194588 12442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 11:16:38.194775 12362 master.cc:537] Master@127.30.255.189:38361 shutting down...
I20260812 11:16:38.195192 12547 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:16:38.196830 11452 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 11:16:38.198482 11452 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d -> Peer 353be5983f544708a027d37acbe885b4 (127.30.255.189:38361): Couldn't send request to peer 353be5983f544708a027d37acbe885b4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 11:16:38.199573 12362 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:38.199834 12362 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:38.199932 12362 tablet_replica.cc:330] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: stopping tablet replica
I20260812 11:16:38.204128 12362 master.cc:552] Master@127.30.255.189:38361 shutdown complete.
I20260812 11:16:38.204175 12362 master_runner.cc:305] Clearing existing system tablet
I20260812 11:16:38.204396 12362 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.204452 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
WARNING: no policy specified for oryx/127.30.255.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.188@KRBTEST.COM" created.
I20260812 11:16:38.210319 12362 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.002s
I20260812 11:16:38.211097 12556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.211297 12362 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.211400 12362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.211431 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "353be5983f544708a027d37acbe885b4"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "3996ad2d23a9bb99a500a6182f72d200"
server_key_iv: "a6c4869be9c0bdd637ff4be51d87ab83"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.213394 12362 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:38.215730 12362 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 11:16:38.215821 12362 log.cc:1196] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 11:16:38.216059 12362 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Deleting consensus metadata
I20260812 11:16:38.216223 12362 master_runner.cc:315] Copying system tablet from 127.30.255.190:38763
I20260812 11:16:38.217258 12362 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: tablet copy: Beginning tablet copy session from remote peer at address 127.30.255.190:38763
I20260812 11:16:38.226191 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.217737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51658)
0812 11:16:38.218028 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.218042 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.218159 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.218399 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.218402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.218612 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.218762 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.218771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.219893 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.219897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.221385 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.221398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.221995 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.221999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.222108 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.222611 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.222625 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.223028 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.224781 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.224784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.224785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.224967 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.225236 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.225238 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.225239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.225307 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.225697 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.225700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.225781 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.225865 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.225992 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":87,"threads_started":1}
I20260812 11:16:38.226227 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.217799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51658 (local address 127.30.255.190:38763)
0812 11:16:38.217915 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.217920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.218131 (+ 211us) server_negotiation.cc:407] Connection header received
0812 11:16:38.218233 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.218236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.218279 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.218361 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.218870 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.219757 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.221527 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.221855 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.221970 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.223194 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.223198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.223201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.223227 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.224683 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.225060 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.225063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.225065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.225158 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.225401 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.225404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.225406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.225585 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.225674 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.225993 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.226106 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:16:38.226670 11522 tablet_copy_service.cc:140] P 7994d6ab0efa484886eb818b2eb0008d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 353be5983f544708a027d37acbe885b4 ({username='oryx', principal='oryx/127.30.255.189@KRBTEST.COM'} at 127.0.0.1:51658)
I20260812 11:16:38.226824 11522 tablet_copy_service.cc:161] P 7994d6ab0efa484886eb818b2eb0008d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 353be5983f544708a027d37acbe885b4 at {username='oryx', principal='oryx/127.30.255.189@KRBTEST.COM'} at 127.0.0.1:51658: session id = 353be5983f544708a027d37acbe885b4-00000000000000000000000000000000
I20260812 11:16:38.228245 11522 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 11:16:38.229642 12362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 11:16:38.232229 12362 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: tablet copy: Starting download of 0 data blocks...
I20260812 11:16:38.232404 12362 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: tablet copy: Starting download of 1 WAL segments...
I20260812 11:16:38.233892 12362 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.188.keytab.
Entry for principal oryx/127.30.255.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.188.keytab.
I20260812 11:16:38.235594 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:44681
--webserver_interface=127.30.255.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.188.keytab
--principal=oryx/127.30.255.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:16:38.236219 11522 tablet_copy_service.cc:342] P 7994d6ab0efa484886eb818b2eb0008d: Request end of tablet copy session 353be5983f544708a027d37acbe885b4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.255.189@KRBTEST.COM'} at 127.0.0.1:51658
I20260812 11:16:38.236330 11522 tablet_copy_service.cc:433] P 7994d6ab0efa484886eb818b2eb0008d: ending tablet copy session 353be5983f544708a027d37acbe885b4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 353be5983f544708a027d37acbe885b4
I20260812 11:16:38.244213 12362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.245865 12570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.245967 12574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.246234 12362 server_base.cc:1034] running on GCE node
W20260812 11:16:38.246546 12571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.246771 12362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.247045 12362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.248173 12362 hybrid_clock.cc:648] HybridClock initialized: now 1786533398248163 us; error 34 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.249711 12362 init.cc:377] Logged in from keytab as oryx/127.30.255.189@KRBTEST.COM (short username oryx)
I20260812 11:16:38.250144 12362 webserver.cc:460] Webserver started at http://127.30.255.189:40789/ using document root <none> and password file <none>
I20260812 11:16:38.250316 12362 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.250355 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.251538 12362 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.252487 12591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.252723 12362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.252796 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "353be5983f544708a027d37acbe885b4"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "3996ad2d23a9bb99a500a6182f72d200"
server_key_iv: "a6c4869be9c0bdd637ff4be51d87ab83"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.252866 12362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.263245 12362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.265167 12362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.270915 12362 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.189:38361
I20260812 11:16:38.271512 12362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
I20260812 11:16:38.271943 12698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.189:38361 every 8 connection(s)
I20260812 11:16:38.273396 12699 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:16:38.274425 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Bootstrap starting.
I20260812 11:16:38.284752 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:16:38.285044 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: Bootstrap complete.
I20260812 11:16:38.285598 12699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } }
I20260812 11:16:38.285739 12699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 353be5983f544708a027d37acbe885b4, State: Initialized, Role: LEARNER
I20260812 11:16:38.285854 12699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } }
I20260812 11:16:38.286196 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } }
I20260812 11:16:38.286310 12699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:38.286399 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:16:38.289234 12721 catalog_manager.cc:1259] Loaded cluster ID: a1fafdfbec5649a498ce79bff06324e7
I20260812 11:16:38.289336 12721 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: loading cluster ID for follower catalog manager: success
I20260812 11:16:38.291462 12721 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: acquiring CA information for follower catalog manager: success
I20260812 11:16:38.292758 12721 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 11:16:38.357275 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:38.357496 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:38.357532 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:38.361373 12569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:38.361419 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:38.361438 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:38.361459 12569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:38.361472 12569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:38.366441 12569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.188.keytab
--principal=oryx/127.30.255.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:44681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.255.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:38.367336 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:38.368238 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.376147 12726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.376168 12729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.376390 12569 server_base.cc:1034] running on GCE node
W20260812 11:16:38.376154 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.376951 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.377955 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.379115 12569 hybrid_clock.cc:648] HybridClock initialized: now 1786533398379103 us; error 36 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.381728 12569 init.cc:377] Logged in from keytab as oryx/127.30.255.188@KRBTEST.COM (short username oryx)
I20260812 11:16:38.382526 12569 webserver.cc:460] Webserver started at http://127.30.255.188:42229/ using document root <none> and password file <none>
I20260812 11:16:38.383028 12569 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.383090 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.383322 12569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:38.385197 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/instance:
uuid: "21adb03acb064157b5540806312e2b84"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "9e799dffb8672e01fb45a50625375024"
server_key_iv: "2b3315e800def1e23ac69921ce0bdb84"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.385720 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal/instance:
uuid: "21adb03acb064157b5540806312e2b84"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "9e799dffb8672e01fb45a50625375024"
server_key_iv: "2b3315e800def1e23ac69921ce0bdb84"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.389550 12569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 11:16:38.391743 12747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.392868 12569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:38.392970 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "21adb03acb064157b5540806312e2b84"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "9e799dffb8672e01fb45a50625375024"
server_key_iv: "2b3315e800def1e23ac69921ce0bdb84"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.393064 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.420751 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.423318 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:38.423475 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.430776 12569 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.188:44681
I20260812 11:16:38.431027 12842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.188:44681 every 8 connection(s)
I20260812 11:16:38.431794 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
I20260812 11:16:38.434590 12845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:38.438220 12845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:16:38.441947 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 12569
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.188@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
I20260812 11:16:38.451515 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.440049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.30.255.190:38763)
0812 11:16:38.440202 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.440206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.441003 (+ 797us) server_negotiation.cc:407] Connection header received
0812 11:16:38.441914 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.441918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.441977 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.442049 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.443314 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.443838 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.444960 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.445185 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.445275 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.447894 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.447898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.447901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.447929 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.449358 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.449956 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.449960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.449962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.450053 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.450443 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.450445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.450446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.450630 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.450703 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.451214 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.451357 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:16:38.452375 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.440316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51660)
0812 11:16:38.440864 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.440906 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.441686 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.442222 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.442229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.442553 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.443143 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.443161 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.443976 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.443979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.444838 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.444848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.445322 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.445326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.445488 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.446010 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.446029 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.447752 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.449455 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.449464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.449474 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.449859 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.450162 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.450165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.450167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.450334 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.450729 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.450735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.450985 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.451166 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.451449 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":153,"threads_started":1}
I20260812 11:16:38.452412 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.442779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.30.255.188:44681)
0812 11:16:38.443104 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.443113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.443132 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:16:38.443166 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.443170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.443356 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.443494 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.444186 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.444933 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.445832 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.446325 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.446429 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.448998 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.449003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.449013 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.449034 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.450730 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.451092 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.451097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.451099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.451184 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.451488 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.451491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.451493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.451757 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.451937 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.452124 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.452257 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":146,"threads_started":1}
I20260812 11:16:38.453655 12845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.188@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.461452 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.454085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46370)
0812 11:16:38.454198 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.454211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.454279 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.454712 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.454715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.454856 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.454943 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.454948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.455912 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.455916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.456442 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.456450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.456916 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.456919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.457016 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.457387 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.457400 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.458514 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.460107 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.460110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.460112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.460301 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.460560 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.460563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.460565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.460642 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.461012 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.461015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.461100 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.461191 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.461310 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 11:16:38.461510 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.454196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.30.255.189:38361)
0812 11:16:38.454501 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.454506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.454519 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:38.454558 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.454560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.454597 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.454679 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.455060 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.455808 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.456571 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.456814 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.456903 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.458653 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.458656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.458658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.458681 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.460003 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.460394 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.460398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.460400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.460476 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.460723 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.460726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.460728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.460919 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.461004 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.461256 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.461368 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":141,"threads_started":1}
I20260812 11:16:38.462181 12845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.188@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
I20260812 11:16:38.468853 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.462581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:60624)
0812 11:16:38.462679 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.462691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.462759 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.462936 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.462939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.463077 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.463144 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.463149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.463849 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.463852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.464325 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.464332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.464740 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.464743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.464812 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.465164 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.465175 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.466194 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.467632 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.467635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.467637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.467806 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.468037 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.468040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.468042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.468113 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.468431 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.468434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.468511 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.468592 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.468703 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 11:16:38.468879 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.462687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.30.255.188:44681)
0812 11:16:38.462775 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.462779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.462792 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:38.462820 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.462822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.462857 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.462915 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.463242 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.463746 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.464424 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.464647 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.464728 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.466310 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.466314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.466316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.466337 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.467537 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.467887 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.467890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.467892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.467958 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.468196 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.468199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.468201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.468345 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.468426 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.468655 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.468761 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":32}
I20260812 11:16:38.471717 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Bootstrap starting.
I20260812 11:16:38.473805 12845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:38.474465 12845 log.cc:824] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:38.476208 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: No bootstrap required, opened a new log
I20260812 11:16:38.478977 12845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } }
I20260812 11:16:38.479161 12845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:38.479223 12845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21adb03acb064157b5540806312e2b84, State: Initialized, Role: FOLLOWER
I20260812 11:16:38.479384 11861 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
I20260812 11:16:38.479689 12845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } }
I20260812 11:16:38.480149 12856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } } }
I20260812 11:16:38.480293 12856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:38.480880 12845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:38.481832 12856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:38.481958 12856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } }
I20260812 11:16:38.483253 11498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21adb03acb064157b5540806312e2b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7994d6ab0efa484886eb818b2eb0008d" is_pre_election: true
I20260812 11:16:38.483336 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21adb03acb064157b5540806312e2b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353be5983f544708a027d37acbe885b4" is_pre_election: true
I20260812 11:16:38.483510 12635 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Handling vote request from an unknown peer 21adb03acb064157b5540806312e2b84
I20260812 11:16:38.483657 12635 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 21adb03acb064157b5540806312e2b84 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 11:16:38.482841 12856 leader_election.cc:288] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763), 353be5983f544708a027d37acbe885b4 (127.30.255.189:38361)
I20260812 11:16:38.484010 12750 leader_election.cc:302] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21adb03acb064157b5540806312e2b84; no voters: 353be5983f544708a027d37acbe885b4, 7994d6ab0efa484886eb818b2eb0008d
I20260812 11:16:38.484262 12856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:38.485213 12856 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 11:16:38.485714 12873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:16:38.485792 12873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:16:38.489161 12569 master_runner.cc:428] Detected that this master 21adb03acb064157b5540806312e2b84 is joining an existing cluster
I20260812 11:16:38.489209 12569 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.255.188:44681
I20260812 11:16:38.498862 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51666)
0812 11:16:38.491520 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.491534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.491677 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.492040 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.492043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.492216 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.492352 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.492360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.493393 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.493396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.494305 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.494312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.494865 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.495028 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.495525 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.495542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.495855 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.497623 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.497626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.497627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.497821 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.498068 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.498069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.498071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.498137 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.498486 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.498488 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.498558 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.498635 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.498731 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":64,"spinlock_wait_cycles":6528,"thread_start_us":98,"threads_started":1}
I20260812 11:16:38.498948 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.30.255.190:38763)
0812 11:16:38.491225 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.491229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.491596 (+ 367us) server_negotiation.cc:407] Connection header received
0812 11:16:38.491795 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.491799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.491847 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.491898 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.492475 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.493285 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494415 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.494756 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494863 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.495981 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.495984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.495987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.496013 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.497513 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.497903 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.497907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.497908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.497994 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.498206 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.498209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.498211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.498410 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.498506 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.498694 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.498801 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:16:38.505748 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11611
I20260812 11:16:38.505985 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.30.255.188:44681)
0812 11:16:38.491772 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.491776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.499165 (+ 7389us) server_negotiation.cc:407] Connection header received
0812 11:16:38.499323 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.499327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.499364 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.499427 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.499841 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.500562 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.501446 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.501657 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.501738 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.502872 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.502875 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.502878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.502903 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.504441 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.504875 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.504878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.504880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.504956 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.505194 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.505197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.505199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.505406 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.505469 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.505745 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.505847 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:38.506330 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:60638)
0812 11:16:38.499115 (+ 7501us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.499127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.499211 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.499451 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.499454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.499611 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.499722 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.499729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.500693 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.500696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.501339 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.501349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.501747 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.501751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.501831 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.502363 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.502376 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.502741 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.504551 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.504554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.504556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.504781 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.505040 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.505042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.505044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.505113 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.505522 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.505524 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.505606 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.506082 (+ 476us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.506192 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7434,"thread_start_us":80,"threads_started":1}
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533397, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.190@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
I20260812 11:16:38.517714 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46382)
0812 11:16:38.491938 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.491951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.492040 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.492287 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.492289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.492478 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.492572 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.492578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.493271 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.493274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.494288 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.494300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.494847 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.494991 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.511898 (+ 16907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.511914 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.512368 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.516432 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.516434 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.516436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.516623 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.516857 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.516859 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.516861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.516925 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.517291 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.517294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.517379 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.517460 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.517568 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":106,"spinlock_wait_cycles":1280,"thread_start_us":79,"threads_started":1}
I20260812 11:16:38.518421 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.491101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.30.255.189:38361)
0812 11:16:38.491214 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.491219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.492032 (+ 813us) server_negotiation.cc:407] Connection header received
0812 11:16:38.492124 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.492127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.492176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.492261 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.492683 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.493170 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494428 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.494656 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.494709 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.512553 (+ 17844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.512558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.512561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.512591 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.516323 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.516711 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.516714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.516716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.516783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.516995 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.516997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.516999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.517176 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.517246 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.518188 (+ 942us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.518300 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:16:38.529798 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.502130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:60648)
0812 11:16:38.502242 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.502255 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.502315 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.502661 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.502665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.502828 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.502918 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.502923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.503892 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.503896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.504414 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.504422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.504951 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.504955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.505041 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.505443 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.505457 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.526387 (+ 20930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.528293 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.528297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.528299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.528498 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.528763 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.528766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.528768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.528889 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.529271 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.529275 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.529384 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.529472 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.529637 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 11:16:38.529811 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.502165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.30.255.188:44681)
0812 11:16:38.502383 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.502391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.502404 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:38.502466 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.502469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.502516 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.503010 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.503090 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.503753 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.504523 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.504830 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.505249 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.526603 (+ 21354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.526606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.526608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.526632 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.528195 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.528590 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.528594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.528595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.528676 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.528974 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.528977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.528979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.529182 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.529267 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.529551 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.529657 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260812 11:16:38.530892 11481 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.255.188:44681: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "21adb03acb064157b5540806312e2b84"
member_type: NON_VOTER
last_known_addr {
host: "127.30.255.188"
port: 44681
}
attrs {
promote: true
}
}
dest_uuid: "7994d6ab0efa484886eb818b2eb0008d"
cas_config_opid_index: 5
I20260812 11:16:38.531491 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } }
I20260812 11:16:38.532546 12547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 21adb03acb064157b5540806312e2b84 (127.30.255.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.533773 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:36147
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:38763
--webserver_port=42501
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:16:38.533972 12547 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.255.188:44681
I20260812 11:16:38.534135 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.534235 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:38.534435 12794 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 11:16:38.534657 12569 master.cc:537] Master@127.30.255.188:44681 shutting down...
I20260812 11:16:38.534992 12549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:16:38.536923 11452 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d -> Peer 21adb03acb064157b5540806312e2b84 (127.30.255.188:44681): Couldn't send request to peer 21adb03acb064157b5540806312e2b84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 11:16:38.540745 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.533676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:46388)
0812 11:16:38.535755 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.535769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.535855 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.537210 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.537214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.537234 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:38.537315 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.537320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.538323 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.538327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.539467 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.539479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.540328 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.540331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.540414 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.540457 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.540532 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.540613 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2008}
I20260812 11:16:38.540796 12569 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:38.541050 12569 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:38.541081 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.534302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.30.255.189:38361)
0812 11:16:38.535615 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.535620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.535823 (+ 203us) server_negotiation.cc:407] Connection header received
0812 11:16:38.536001 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.536004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.536089 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.536216 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:38.537426 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.538188 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.539610 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.540221 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.540335 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.540477 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.540899 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.540972 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229}
I20260812 11:16:38.541179 12569 tablet_replica.cc:330] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: stopping tablet replica
I20260812 11:16:38.541981 12635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 11:16:38.542421 12549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 11:16:38.544191 12635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 21adb03acb064157b5540806312e2b84 (127.30.255.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.545912 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7994d6ab0efa484886eb818b2eb0008d. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.546007 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:16:38.546096 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: NON_VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: true } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.546167 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:16:38.546790 12549 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: attempting to promote NON_VOTER 353be5983f544708a027d37acbe885b4 to VOTER
I20260812 11:16:38.547210 12549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } }
I20260812 11:16:38.548849 12635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 LEARNER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 11:16:38.549748 12547 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 11:16:38.550043 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.549559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:60656)
0812 11:16:38.549904 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.549974 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
W20260812 11:16:38.550304 11452 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d -> Peer 21adb03acb064157b5540806312e2b84 (127.30.255.188:44681): Couldn't send request to peer 21adb03acb064157b5540806312e2b84. Status: Network error: Client connection negotiation failed: client connection to 127.30.255.188:44681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 11:16:38.551522 12547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 353be5983f544708a027d37acbe885b4 (127.30.255.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.552711 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.552815 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:38.552943 12547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.553027 12547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:38.553577 12635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 353be5983f544708a027d37acbe885b4 (127.30.255.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.554900 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.555028 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:38.561131 12569 master.cc:552] Master@127.30.255.188:44681 shutdown complete.
I20260812 11:16:38.561177 12569 master_runner.cc:305] Clearing existing system tablet
I20260812 11:16:38.561380 12569 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.561429 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.562436 12569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.563166 12898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.563395 12569 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.563514 12569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.563552 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "21adb03acb064157b5540806312e2b84"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "9e799dffb8672e01fb45a50625375024"
server_key_iv: "2b3315e800def1e23ac69921ce0bdb84"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.565153 12569 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:38.566998 12569 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 11:16:38.567078 12569 log.cc:1196] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 11:16:38.567375 12569 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Deleting consensus metadata
I20260812 11:16:38.567543 12569 master_runner.cc:315] Copying system tablet from 127.30.255.190:38763
I20260812 11:16:38.568532 12569 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: tablet copy: Beginning tablet copy session from remote peer at address 127.30.255.190:38763
I20260812 11:16:38.576099 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.569047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.30.255.190:38763)
0812 11:16:38.569172 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.569176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.569379 (+ 203us) server_negotiation.cc:407] Connection header received
0812 11:16:38.569484 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.569487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.569530 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.569596 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.570014 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.570661 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.571688 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.571912 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.571998 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.573224 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.573228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.573231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.573256 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.574531 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.574928 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.574932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.574933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.575011 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.575307 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.575311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.575313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.575496 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.575594 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.575847 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.575946 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:16:38.576109 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.569004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:51676)
0812 11:16:38.569294 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.569305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.569407 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.569622 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.569625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.569811 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.569905 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.569910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.570760 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.570764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.571493 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.571504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.572025 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.572029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.572114 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.572761 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.572773 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.573083 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.574638 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.574641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.574643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.574840 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.575095 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.575098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.575100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.575191 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.575600 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.575604 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.575696 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.575783 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.575922 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":86,"threads_started":1}
I20260812 11:16:38.576700 11522 tablet_copy_service.cc:140] P 7994d6ab0efa484886eb818b2eb0008d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 21adb03acb064157b5540806312e2b84 ({username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:51676)
I20260812 11:16:38.576781 11522 tablet_copy_service.cc:161] P 7994d6ab0efa484886eb818b2eb0008d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 21adb03acb064157b5540806312e2b84 at {username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:51676: session id = 21adb03acb064157b5540806312e2b84-00000000000000000000000000000000
I20260812 11:16:38.577520 11522 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 11:16:38.578538 12569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 11:16:38.580204 12569 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: tablet copy: Starting download of 0 data blocks...
I20260812 11:16:38.580370 12569 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: tablet copy: Starting download of 1 WAL segments...
I20260812 11:16:38.581606 12569 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 11:16:38.583030 11522 tablet_copy_service.cc:342] P 7994d6ab0efa484886eb818b2eb0008d: Request end of tablet copy session 21adb03acb064157b5540806312e2b84-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:51676
I20260812 11:16:38.583098 11522 tablet_copy_service.cc:433] P 7994d6ab0efa484886eb818b2eb0008d: ending tablet copy session 21adb03acb064157b5540806312e2b84-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 21adb03acb064157b5540806312e2b84
I20260812 11:16:38.589395 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.591033 12909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.591040 12914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.591442 12911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.595476 12569 server_base.cc:1034] running on GCE node
I20260812 11:16:38.595634 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.595894 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.597024 12569 hybrid_clock.cc:648] HybridClock initialized: now 1786533398597017 us; error 23 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.598285 12569 init.cc:377] Logged in from keytab as oryx/127.30.255.188@KRBTEST.COM (short username oryx)
I20260812 11:16:38.598703 12569 webserver.cc:460] Webserver started at http://127.30.255.188:43791/ using document root <none> and password file <none>
I20260812 11:16:38.598874 12569 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.598909 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.599965 12569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.600791 12921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.601003 12569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:38.601074 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "21adb03acb064157b5540806312e2b84"
format_stamp: "Formatted at 2026-08-12 11:16:38 on dist-test-slave-gjrz"
server_key: "9e799dffb8672e01fb45a50625375024"
server_key_iv: "2b3315e800def1e23ac69921ce0bdb84"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.601161 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.623580 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.625289 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.630247 13018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.188:44681 every 8 connection(s)
I20260812 11:16:38.630228 12569 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.188:44681
I20260812 11:16:38.630824 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
I20260812 11:16:38.632249 13020 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:16:38.633177 13020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Bootstrap starting.
I20260812 11:16:38.641659 13020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:16:38.641909 13020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Bootstrap complete.
I20260812 11:16:38.642355 13020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } }
I20260812 11:16:38.642498 13020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 21adb03acb064157b5540806312e2b84, State: Initialized, Role: LEARNER
I20260812 11:16:38.642611 13020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } }
I20260812 11:16:38.642937 13025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:38.643076 13020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:38.643138 13025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:16:38.646747 13052 catalog_manager.cc:1259] Loaded cluster ID: a1fafdfbec5649a498ce79bff06324e7
I20260812 11:16:38.646797 13052 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: loading cluster ID for follower catalog manager: success
I20260812 11:16:38.648365 13052 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: acquiring CA information for follower catalog manager: success
I20260812 11:16:38.649502 13052 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:16:38.686020 12110 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
W20260812 11:16:38.687793 12890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:38.688005 12890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:38.688045 12890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:38.691891 12890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:38.691936 12890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:38.691989 12890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:38.696905 12890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:36147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=42501
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:38.697777 12890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:38.698665 12890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.707944 13059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.708020 13067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.708086 13060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.708508 12890 server_base.cc:1034] running on GCE node
I20260812 11:16:38.708868 12890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.709429 12890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.710574 12890 hybrid_clock.cc:648] HybridClock initialized: now 1786533398710548 us; error 39 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.713218 12890 init.cc:377] Logged in from keytab as oryx/127.30.255.129@KRBTEST.COM (short username oryx)
I20260812 11:16:38.714020 12890 webserver.cc:460] Webserver started at http://127.30.255.129:42501/ using document root <none> and password file <none>
I20260812 11:16:38.714524 12890 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.714601 12890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.718152 12890 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:38.720058 13081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.721163 12890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:38.721268 12890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "84717fdae41747658340f4b452e62248"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "26b5f78d654ae4a0efba1adeed16aa5c"
server_key_iv: "2399f4e588ba36973f04526904e24aa0"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.721745 12890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.737393 12890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.740036 12890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:38.740180 12890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.740659 12890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:38.741412 12890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:38.741458 12890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.741493 12890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:38.741513 12890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.751164 12890 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:36147
I20260812 11:16:38.751195 13310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:36147 every 8 connection(s)
I20260812 11:16:38.752103 12890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:38.754227 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 12890
I20260812 11:16:38.754410 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11873
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:38.774057 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.753852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:35271 (local address 127.30.255.190:38763)
0812 11:16:38.754512 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.754517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.760206 (+ 5689us) server_negotiation.cc:407] Connection header received
0812 11:16:38.761290 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.761294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.761374 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.761487 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.762794 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.763907 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.766249 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.766519 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.766614 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.769651 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.769655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.769658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.769685 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.771357 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.771934 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.771938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.771940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.772026 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.772388 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.772391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.772393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.772634 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.772709 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.773761 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.773885 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565}
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.775933 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.754352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:35271)
0812 11:16:38.760061 (+ 5709us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.760113 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.761050 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.761638 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.761646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.762056 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.762637 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.762653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.764095 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.764100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.766106 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.766132 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.766766 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.766769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.766948 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.767680 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.767700 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.769450 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.771479 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.771485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.771500 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.771819 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.772128 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.772131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.772133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.772290 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.773182 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.773189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.773484 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.773691 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.773992 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5510,"thread_start_us":140,"threads_started":1}
I20260812 11:16:38.775991 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:35291
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:38763
--webserver_port=41859
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:16:38.777220 13313 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:38.777494 13313 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.778301 13313 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
I20260812 11:16:38.779681 11481 ts_manager.cc:194] Re-registered known tserver with Master: 84717fdae41747658340f4b452e62248 (127.30.255.129:36147)
I20260812 11:16:38.780393 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.754352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.30.255.129:39747)
0812 11:16:38.768101 (+ 13749us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.768117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.768311 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.768680 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.768682 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.768984 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.769111 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.769118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.770546 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.770550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.771680 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.771692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.772414 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.772417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.772566 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.773111 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.773125 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.774890 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.778581 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.778583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.778585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.778951 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.779314 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.779317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.779319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.779407 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.779899 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.779901 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.779999 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.780086 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.780226 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13658,"mutex_wait_us":202,"spinlock_wait_cycles":28800,"thread_start_us":125,"threads_started":1}
I20260812 11:16:38.780416 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.760136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:46647 (local address 127.30.255.189:38361)
0812 11:16:38.767566 (+ 7430us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.767571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.768197 (+ 626us) server_negotiation.cc:407] Connection header received
0812 11:16:38.768426 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.768429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.768477 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.768532 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.769406 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.770423 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.771818 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.772167 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.772244 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.775792 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.775796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.775799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.775827 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.777528 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.778994 (+ 1466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.778998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.779000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.779113 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.779458 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.779462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.779463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.779682 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.779771 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.780155 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.780272 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7329}
I20260812 11:16:38.780388 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:35271
I20260812 11:16:38.780995 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.755383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.30.255.129:46647)
0812 11:16:38.768101 (+ 12718us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.768129 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.768311 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.768675 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.768678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.769158 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.769283 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.769289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.770546 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.770550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.771684 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.771692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.772400 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.772404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.772539 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.773171 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.773184 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.775634 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.777717 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.777721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.777724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.778876 (+ 1152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.779223 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.779226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.779228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.779350 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.779858 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.779860 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.779946 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.780690 (+ 744us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.780832 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12665,"mutex_wait_us":115,"spinlock_wait_cycles":29568,"thread_start_us":4428,"threads_started":1}
I20260812 11:16:38.781771 13318 heartbeater.cc:347] Connected to a master server at 127.30.255.189:38361
I20260812 11:16:38.781858 13318 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.781852 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.753853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:39747 (local address 127.30.255.188:44681)
0812 11:16:38.754116 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.754139 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.768200 (+ 14061us) server_negotiation.cc:407] Connection header received
0812 11:16:38.768427 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.768430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.768481 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.768536 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.769265 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.770370 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.771818 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.772176 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.772244 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.775057 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.775061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.775064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.775093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.778423 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.779075 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.779079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.779081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.779188 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.779506 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.779509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.779511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.779718 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.779808 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.781565 (+ 1757us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.781700 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260812 11:16:38.782021 13318 heartbeater.cc:511] Master 127.30.255.189:38361 requested a full tablet report, sending...
I20260812 11:16:38.782532 13311 heartbeater.cc:347] Connected to a master server at 127.30.255.188:44681
I20260812 11:16:38.782622 13311 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.782793 13311 heartbeater.cc:511] Master 127.30.255.188:44681 requested a full tablet report, sending...
I20260812 11:16:38.783236 12625 ts_manager.cc:194] Registered new tserver with Master: 84717fdae41747658340f4b452e62248 (127.30.255.129:36147)
I20260812 11:16:38.783843 12946 ts_manager.cc:194] Registered new tserver with Master: 84717fdae41747658340f4b452e62248 (127.30.255.129:36147)
W20260812 11:16:38.888809 13326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:38.889036 13326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:38.889078 13326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:38.893025 13326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:38.893071 13326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:38.893141 13326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:38.896445 12355 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
I20260812 11:16:38.898039 13326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:35291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=41859
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:38.898862 13326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:38.899762 13326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:38.908916 13333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:38.908969 13334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.909034 13326 server_base.cc:1034] running on GCE node
W20260812 11:16:38.908968 13338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:38.909574 13326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:38.910110 13326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:38.911252 13326 hybrid_clock.cc:648] HybridClock initialized: now 1786533398911224 us; error 44 us; skew 500 ppm
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:38.913790 13326 init.cc:377] Logged in from keytab as oryx/127.30.255.130@KRBTEST.COM (short username oryx)
I20260812 11:16:38.914557 13326 webserver.cc:460] Webserver started at http://127.30.255.130:41859/ using document root <none> and password file <none>
I20260812 11:16:38.915061 13326 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:38.915128 13326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:38.918584 13326 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:16:38.920512 13353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.921609 13326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:38.921708 13326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "83f4c764308448059050d97df4271080"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "d7ce0d65f2cccaa012d39b581161c1a2"
server_key_iv: "bf032f16c42a65d5a777480e70381191"
server_key_version: "encryptionkey@0"
I20260812 11:16:38.922206 13326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:38.940917 13326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:38.943385 13326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:38.943538 13326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:38.944046 13326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:38.944809 13326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:38.944855 13326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.944890 13326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:38.944917 13326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:38.953657 13326 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:35291
I20260812 11:16:38.953683 13567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:35291 every 8 connection(s)
I20260812 11:16:38.954538 13326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:38.963094 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 13326
I20260812 11:16:38.963289 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 12122
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
Aug 12 11:16:38 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533398, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
I20260812 11:16:38.968598 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.957898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:50891 (local address 127.30.255.188:44681)
0812 11:16:38.958011 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.958016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.958105 (+ 89us) server_negotiation.cc:407] Connection header received
0812 11:16:38.958540 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.958543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.958606 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.958678 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.959914 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.960722 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962099 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962385 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962484 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.965034 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.965038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.965039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.965063 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.966219 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.966797 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.966800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.966802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.966881 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.967223 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.967227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.967228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.967471 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.967570 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.968300 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.968434 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:16:38.970364 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.957767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.30.255.130:50891)
0812 11:16:38.958011 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.958024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.958348 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.958843 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.958849 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.959220 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.959761 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.959776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.961173 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.961176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.961995 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.962003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962633 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.962735 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.963386 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.963399 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.964880 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.966360 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.966365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.966378 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.966699 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.966968 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.966971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.966973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.967103 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.967622 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.967628 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.968015 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.969094 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.969412 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":58,"thread_start_us":82,"threads_started":1}
I20260812 11:16:38.971525 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.956244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:39303 (local address 127.30.255.189:38361)
0812 11:16:38.956367 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.956370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.957418 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 11:16:38.958590 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.958594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.958643 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.958691 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.959934 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.960446 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.961944 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962263 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962358 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.968162 (+ 5804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.968166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.968169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.968197 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.969653 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970172 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.970175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.970177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.970270 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970595 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.970598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.970600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.970811 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.970877 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.971236 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.971381 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:16:38.971665 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.956243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37093 (local address 127.30.255.190:38763)
0812 11:16:38.956443 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:38.956446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:38.957416 (+ 970us) server_negotiation.cc:407] Connection header received
0812 11:16:38.958580 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:38.958584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:38.958639 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:38.958691 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:38.959933 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.960732 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.961828 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962146 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962249 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.967727 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:38.967731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:38.967733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:38.967762 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:38.969733 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970289 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.970292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.970294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.970385 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970755 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:38.970758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:38.970760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:38.971002 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:38.971070 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:38.971415 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 11:16:38.971519 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260812 11:16:38.971709 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.956753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:37093)
0812 11:16:38.957261 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.957319 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.958345 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.958842 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.958851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.959312 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.959772 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.959782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.960838 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.960841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.961693 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.961703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962280 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.962460 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.963123 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.963140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.967581 (+ 4441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.969866 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.969869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.969871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.970147 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.970557 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.970562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.970651 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.971142 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.971144 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.971232 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.971358 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.971494 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":148,"spinlock_wait_cycles":46336,"thread_start_us":147,"threads_started":1}
I20260812 11:16:38.971992 13575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:38.956754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.30.255.130:39303)
0812 11:16:38.957261 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:38.957319 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:38.958345 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:38.958842 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:38.958851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:38.959312 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:38.959775 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.959782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.960604 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.960607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.961811 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:38.961821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:38.962469 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:38.962472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:38.962544 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:38.963084 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:38.963113 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:38.967991 (+ 4878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:38.969772 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.969776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.969778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.970039 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.970402 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:38.970405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:38.970407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:38.970493 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:38.970946 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:38.970948 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:38.971046 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 11:16:38.971699 (+ 653us) client_negotiation.cc:240] Negotiation successful
0812 11:16:38.971832 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":83,"spinlock_wait_cycles":48768,"thread_start_us":139,"threads_started":1}
I20260812 11:16:38.973023 13572 heartbeater.cc:347] Connected to a master server at 127.30.255.189:38361
I20260812 11:16:38.973245 13572 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.973227 13570 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:38.973320 13569 heartbeater.cc:347] Connected to a master server at 127.30.255.188:44681
I20260812 11:16:38.973383 13569 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.973330 13570 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:38.973755 13570 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
I20260812 11:16:38.973757 13572 heartbeater.cc:511] Master 127.30.255.189:38361 requested a full tablet report, sending...
I20260812 11:16:38.973755 13569 heartbeater.cc:511] Master 127.30.255.188:44681 requested a full tablet report, sending...
I20260812 11:16:38.974813 11481 ts_manager.cc:194] Re-registered known tserver with Master: 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:38.975075 12946 ts_manager.cc:194] Registered new tserver with Master: 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:38.975492 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:37093
I20260812 11:16:38.976052 12625 ts_manager.cc:194] Registered new tserver with Master: 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:38.977432 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:39277
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:38763
--webserver_port=43707
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:16:39.086632 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.082734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:60658)
0812 11:16:39.082957 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.082972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.083077 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.083385 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.083387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.083400 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.083501 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.083506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.084484 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.084486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.085248 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.085256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.086253 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.086257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.086337 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.086361 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.086430 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.086507 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":61,"threads_started":1}
I20260812 11:16:39.086649 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.082838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.30.255.188:44681)
0812 11:16:39.082970 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.082975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.083042 (+ 67us) server_negotiation.cc:407] Connection header received
0812 11:16:39.083169 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.083172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.083237 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.083380 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.083594 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.084394 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.085363 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.086109 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.086228 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.086359 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.086453 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.086523 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 11:16:39.087409 13584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:39.087661 13584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260812 11:16:39.087630 12969 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
W20260812 11:16:39.087730 13584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:16:39.087992 13025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7994d6ab0efa484886eb818b2eb0008d. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:39.088143 13025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: LEARNER
W20260812 11:16:39.091897 13584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:39.091948 13584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:39.092028 13584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:39.097013 13584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:39277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=43707
--tserver_master_addrs=127.30.255.190:38763,127.30.255.189:38361,127.30.255.188:44681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:39.097934 13584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:39.098879 13584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:39.108220 13594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:39.108283 13584 server_base.cc:1034] running on GCE node
W20260812 11:16:39.108213 13597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:39.108217 13599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:39.108776 13584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:39.109361 13584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:39.110510 13584 hybrid_clock.cc:648] HybridClock initialized: now 1786533399110486 us; error 36 us; skew 500 ppm
Aug 12 11:16:39 dist-test-slave-gjrz krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533399, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:39.113008 13584 init.cc:377] Logged in from keytab as oryx/127.30.255.131@KRBTEST.COM (short username oryx)
I20260812 11:16:39.113854 13584 webserver.cc:460] Webserver started at http://127.30.255.131:43707/ using document root <none> and password file <none>
I20260812 11:16:39.114336 13584 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:39.114403 13584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:39.117938 13584 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:39.119824 13617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:39.120918 13584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:39.121012 13584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "fd06182d8bc84772926e4aecaa712379"
format_stamp: "Formatted at 2026-08-12 11:16:37 on dist-test-slave-gjrz"
server_key: "6dc9597deaa805eb9f7fe1b3c75d778f"
server_key_iv: "73a1f197edec6e6a38591f4a53c041c5"
server_key_version: "encryptionkey@0"
I20260812 11:16:39.121490 13584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:39.140712 13584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:39.143566 13584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:39.143731 13584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:39.144297 13584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:39.145289 13584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:39.145340 13584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:39.145375 13584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:39.145390 13584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:39.153703 13584 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:39277
I20260812 11:16:39.153734 13812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:39277 every 8 connection(s)
I20260812 11:16:39.154618 13584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:39.163292 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 13584
Aug 12 11:16:39 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533399, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
Aug 12 11:16:39 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533399, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.189@KRBTEST.COM
Aug 12 11:16:39 dist-test-slave-gjrz krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533399, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.188@KRBTEST.COM
I20260812 11:16:39.170493 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.156270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:44333 (local address 127.30.255.190:38763)
0812 11:16:39.156382 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.156386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.157865 (+ 1479us) server_negotiation.cc:407] Connection header received
0812 11:16:39.158628 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.158632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.158682 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.158744 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.160053 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160846 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.161841 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162104 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162193 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.166120 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.166124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.166127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.166158 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.167992 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168766 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.168770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.168771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.168847 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.169279 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.169282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.169284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.169506 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.169588 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.170196 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.170316 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:39.170529 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.157924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:50207 (local address 127.30.255.188:44681)
0812 11:16:39.158045 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.158049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.158154 (+ 105us) server_negotiation.cc:407] Connection header received
0812 11:16:39.158563 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.158566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.158627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.158679 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.160053 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160854 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162524 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162810 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162915 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.165733 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.165737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.165739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.165769 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.167914 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168750 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.168754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.168756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.168847 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.169278 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.169282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.169283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.169506 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.169591 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.170308 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.170409 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:16:39.170935 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.157958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:37063 (local address 127.30.255.189:38361)
0812 11:16:39.158080 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.158085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.158101 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:16:39.158534 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.158537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.158593 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.158668 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.160034 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160556 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162056 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162353 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162430 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.165753 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.165758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.165760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.165786 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.168120 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168757 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.168761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.168763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.168851 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.169251 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.169254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.169256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.169453 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.169547 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.170662 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.170788 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:16:39.171985 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.156795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.30.255.131:37063)
0812 11:16:39.157260 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.157299 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.158339 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.158841 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.158848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.159328 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:39.159895 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.159909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160702 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.160706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.161945 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.161954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162457 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.162546 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.163355 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:39.163370 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:39.165320 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:39.168238 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.168653 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.168953 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.169146 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.169599 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:39.169605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:39.169896 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.170113 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.171528 (+ 1415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":101,"spinlock_wait_cycles":73600,"thread_start_us":133,"threads_started":1}
I20260812 11:16:39.172240 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.156795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:44333)
0812 11:16:39.157781 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.157794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.158342 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.158845 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.158851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.159347 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:39.159909 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.159918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160985 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.160988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.161728 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.161737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162238 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.162378 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.162958 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:39.162978 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:39.165961 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:39.168107 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.168635 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.168949 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.169169 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.169647 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:39.169649 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:39.169911 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.170734 (+ 823us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.170964 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":184,"spinlock_wait_cycles":162560,"thread_start_us":150,"threads_started":1}
I20260812 11:16:39.173377 13820 heartbeater.cc:347] Connected to a master server at 127.30.255.189:38361
I20260812 11:16:39.173585 13820 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:39.174111 13820 heartbeater.cc:511] Master 127.30.255.189:38361 requested a full tablet report, sending...
I20260812 11:16:39.174628 13818 heartbeater.cc:347] Connected to a master server at 127.30.255.190:38763
I20260812 11:16:39.174705 13818 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:39.174858 13818 heartbeater.cc:511] Master 127.30.255.190:38763 requested a full tablet report, sending...
I20260812 11:16:39.175567 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.157806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.30.255.131:50207)
0812 11:16:39.158075 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.158087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.158353 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.158843 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.158851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.159347 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:39.159908 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.159918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.160951 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.160954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.161943 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.161954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.162943 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.162947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.163065 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.163783 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:39.163797 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:39.165602 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:39.168046 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168067 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.168635 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.168949 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:39.168952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:39.168955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:39.169169 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:39.169636 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:39.169639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:39.169895 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.170104 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.173495 (+ 3391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":3321,"spinlock_wait_cycles":31104,"thread_start_us":73,"threads_started":1}
I20260812 11:16:39.175665 12625 ts_manager.cc:194] Registered new tserver with Master: fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:39.175694 11481 ts_manager.cc:194] Re-registered known tserver with Master: fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:39.176245 13813 heartbeater.cc:347] Connected to a master server at 127.30.255.188:44681
I20260812 11:16:39.176316 13813 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:39.176365 11481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:44333
I20260812 11:16:39.176397 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.165514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.30.255.188:44681)
0812 11:16:39.166098 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.166102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.166483 (+ 381us) server_negotiation.cc:407] Connection header received
0812 11:16:39.166525 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.166528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.166577 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.166660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.167685 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.168413 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.169539 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.169862 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.169955 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.171871 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.171875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.171878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.171902 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.173769 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.174212 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.174215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.174217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.174303 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.174618 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.174621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.174623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.174836 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.174946 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.176132 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.176253 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":79,"threads_started":1}
I20260812 11:16:39.176469 13813 heartbeater.cc:511] Master 127.30.255.188:44681 requested a full tablet report, sending...
I20260812 11:16:39.177196 12946 ts_manager.cc:194] Registered new tserver with Master: fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:39.181072 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.165573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.30.255.189:38361)
0812 11:16:39.171088 (+ 5515us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.171092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.171105 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:39.171138 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.171140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.171179 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.171259 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.172037 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.172658 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.174469 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.174764 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.174839 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.177993 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.177997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.178000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.178023 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.179522 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.179936 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.179938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.179939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.180010 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.180306 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.180310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.180311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.180496 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.180593 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.180837 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.180921 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5461,"thread_start_us":80,"threads_started":1}
I20260812 11:16:39.182667 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.166128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51690 (local address 127.30.255.190:38763)
0812 11:16:39.170484 (+ 4356us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.170489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.173868 (+ 3379us) server_negotiation.cc:407] Connection header received
0812 11:16:39.173906 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.173909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.173956 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.174047 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.174792 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.175708 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.178208 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.178425 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.178494 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.179841 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.179845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.179847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.179872 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.181383 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.181799 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.181801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.181802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.181874 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.182067 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.182069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.182070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.182210 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.182300 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.182432 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.182517 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4284,"thread_start_us":77,"threads_started":1}
I20260812 11:16:39.185009 11481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:39.188048 11481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:39.190402 13853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: NON_VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: true } } }
I20260812 11:16:39.190533 13853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:39.193742 13590 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: attempting to promote NON_VOTER 21adb03acb064157b5540806312e2b84 to VOTER
I20260812 11:16:39.194103 13590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } }
I20260812 11:16:39.195293 12635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 11:16:39.195760 12970 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 LEARNER]: Refusing update from remote peer 7994d6ab0efa484886eb818b2eb0008d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 11:16:39.195752 13862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 11:16:39.196099 13862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 11:16:39.197324 13854 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 21adb03acb064157b5540806312e2b84 (127.30.255.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.198815 13854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.198917 13854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:39.198875 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.198979 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:39.198701 12970 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 21adb03acb064157b5540806312e2b84 (127.30.255.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.199188 13853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.199357 13853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:39.199679 12635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 21adb03acb064157b5540806312e2b84 (127.30.255.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.200760 13025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.200867 13025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:39.201642 13865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7994d6ab0efa484886eb818b2eb0008d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:39.201824 13865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:39.209077 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.203659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39277 (local address 127.0.0.1:55260)
0812 11:16:39.204144 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.204159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.204236 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.205297 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.205300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.205317 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.205396 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.205400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.206621 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.206624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.207437 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.207448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.208703 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.208706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.208800 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.208822 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.208889 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.208961 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406}
I20260812 11:16:39.209565 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.203847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.30.255.131:39277)
0812 11:16:39.204862 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.204874 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.204895 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:16:39.204965 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.204970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.205161 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.205309 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.205530 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.206531 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.207574 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.208393 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.208499 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.208634 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.209340 (+ 706us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.209436 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":119,"threads_started":1}
I20260812 11:16:39.212011 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.204131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36147 (local address 127.0.0.1:47236)
0812 11:16:39.207432 (+ 3301us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.207446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.207614 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.208231 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.208234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.208253 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.208339 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.208345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.209663 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.209666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.210647 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.210658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.211645 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.211649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.211735 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.211756 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.211823 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.211891 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3238,"thread_start_us":88,"threads_started":1}
I20260812 11:16:39.212204 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.204514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.30.255.129:36147)
0812 11:16:39.204799 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.204810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.207543 (+ 2733us) server_negotiation.cc:407] Connection header received
0812 11:16:39.207770 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.207776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.207945 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.208057 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.208473 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.209511 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.210763 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.211335 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.211455 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.211579 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.211871 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.211968 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260812 11:16:39.213104 13698 tablet_service.cc:1467] Processing CreateTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb (DEFAULT_TABLE table=test-table [id=37f01626d7e94a6c8e3fc275fccca052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:39.214151 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 181f6a28bb284e6ab254bb4983a59aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:39.214345 13173 tablet_service.cc:1467] Processing CreateTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb (DEFAULT_TABLE table=test-table [id=37f01626d7e94a6c8e3fc275fccca052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:39.215332 13173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 181f6a28bb284e6ab254bb4983a59aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:39.215819 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.203888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:35291 (local address 127.0.0.1:49476)
0812 11:16:39.205526 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.205539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.205617 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.206212 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.206215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.206231 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.206308 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.206315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.207696 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.207699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.208889 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.208900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.210332 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.210335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.215529 (+ 5194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.215554 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.215629 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.215693 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"thread_start_us":92,"threads_started":1}
I20260812 11:16:39.216235 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.204317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.30.255.130:35291)
0812 11:16:39.204692 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.204703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.205707 (+ 1004us) server_negotiation.cc:407] Connection header received
0812 11:16:39.205787 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.205792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.205965 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.206091 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.206412 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.207565 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.209022 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.210071 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.210174 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.210275 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.215993 (+ 5718us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.216087 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":98,"threads_started":1}
I20260812 11:16:39.219029 13443 tablet_service.cc:1467] Processing CreateTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb (DEFAULT_TABLE table=test-table [id=37f01626d7e94a6c8e3fc275fccca052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:39.220052 13443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 181f6a28bb284e6ab254bb4983a59aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:39.222604 13878 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Bootstrap starting.
I20260812 11:16:39.223755 13880 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Bootstrap starting.
I20260812 11:16:39.224905 13878 tablet_bootstrap.cc:654] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:39.225916 13878 log.cc:824] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:39.226120 13880 tablet_bootstrap.cc:654] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:39.226958 13882 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Bootstrap starting.
I20260812 11:16:39.227195 13880 log.cc:824] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:39.229389 13878 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: No bootstrap required, opened a new log
I20260812 11:16:39.229391 13880 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: No bootstrap required, opened a new log
I20260812 11:16:39.229557 13880 ts_tablet_manager.cc:1397] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 11:16:39.229562 13878 ts_tablet_manager.cc:1397] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:16:39.232066 13882 tablet_bootstrap.cc:654] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:39.232852 13882 log.cc:824] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:39.232812 13880 raft_consensus.cc:348] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.233021 13880 raft_consensus.cc:374] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:39.233083 13880 raft_consensus.cc:729] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84717fdae41747658340f4b452e62248, State: Initialized, Role: FOLLOWER
I20260812 11:16:39.232964 13878 raft_consensus.cc:348] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.233215 13878 raft_consensus.cc:374] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:39.233279 13878 raft_consensus.cc:729] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd06182d8bc84772926e4aecaa712379, State: Initialized, Role: FOLLOWER
I20260812 11:16:39.233533 13880 consensus_queue.cc:260] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.233687 13878 consensus_queue.cc:260] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.234501 13880 ts_tablet_manager.cc:1428] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 11:16:39.234527 13878 ts_tablet_manager.cc:1428] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:16:39.234771 13882 tablet_bootstrap.cc:492] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: No bootstrap required, opened a new log
I20260812 11:16:39.234931 13882 ts_tablet_manager.cc:1397] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 11:16:39.235071 13313 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
I20260812 11:16:39.237673 13818 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
I20260812 11:16:39.239311 13882 raft_consensus.cc:348] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.239521 13882 raft_consensus.cc:374] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:39.239571 13882 raft_consensus.cc:729] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f4c764308448059050d97df4271080, State: Initialized, Role: FOLLOWER
I20260812 11:16:39.239949 13882 consensus_queue.cc:260] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.240866 13882 ts_tablet_manager.cc:1428] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:16:39.241981 13570 heartbeater.cc:503] Master 127.30.255.190:38763 was elected leader, sending a full tablet report...
W20260812 11:16:39.253968 13319 tablet.cc:2369] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:39.405866 13824 tablet.cc:2369] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:39.455855 13574 tablet.cc:2369] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:39.683501 13888 raft_consensus.cc:482] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:39.683723 13888 raft_consensus.cc:504] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.684861 13888 leader_election.cc:288] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277), 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:39.688827 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.685189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:54747 (local address 127.30.255.131:39277)
0812 11:16:39.685320 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.685324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.685426 (+ 102us) server_negotiation.cc:407] Connection header received
0812 11:16:39.685605 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.685609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.685660 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.685701 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.686106 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.686693 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.687933 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.688328 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.688436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.688492 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.688643 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.688718 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:39.688850 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.685084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39277 (local address 127.30.255.129:54747)
0812 11:16:39.685319 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.685334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.685465 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.685825 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.685827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.685843 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.685997 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.686003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.686805 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.686809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.687816 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.687826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.688436 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.688440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.688534 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.688553 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.688618 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.688689 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":76,"thread_start_us":70,"threads_started":1}
I20260812 11:16:39.688884 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.685541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:35651 (local address 127.30.255.130:35291)
0812 11:16:39.685708 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.685712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.685726 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:39.685784 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.685786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.685828 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.685891 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.686131 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.686710 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.687962 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.688396 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.688500 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.688561 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.688712 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.688776 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 11:16:39.688899 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.685418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:35291 (local address 127.30.255.129:35651)
0812 11:16:39.685602 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.685617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.685711 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.685907 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.685910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.685926 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.686027 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.686032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.686814 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.686817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.687842 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.687852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.688500 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.688504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.688600 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.688623 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.688688 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.688758 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":32,"thread_start_us":55,"threads_started":1}
I20260812 11:16:39.689198 13890 raft_consensus.cc:482] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:39.689360 13890 raft_consensus.cc:504] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.689541 13742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "84717fdae41747658340f4b452e62248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd06182d8bc84772926e4aecaa712379" is_pre_election: true
I20260812 11:16:39.689534 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "84717fdae41747658340f4b452e62248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f4c764308448059050d97df4271080" is_pre_election: true
I20260812 11:16:39.689745 13742 raft_consensus.cc:2454] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84717fdae41747658340f4b452e62248 in term 0.
I20260812 11:16:39.689798 13491 raft_consensus.cc:2454] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84717fdae41747658340f4b452e62248 in term 0.
I20260812 11:16:39.690205 13083 leader_election.cc:302] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84717fdae41747658340f4b452e62248, fd06182d8bc84772926e4aecaa712379; no voters:
I20260812 11:16:39.690429 13890 leader_election.cc:288] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f4c764308448059050d97df4271080 (127.30.255.130:35291), 84717fdae41747658340f4b452e62248 (127.30.255.129:36147)
I20260812 11:16:39.690722 13888 raft_consensus.cc:2781] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:39.690812 13888 raft_consensus.cc:482] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:39.690862 13888 raft_consensus.cc:3037] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:39.692126 13888 raft_consensus.cc:504] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.692440 13888 leader_election.cc:288] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [CANDIDATE]: Term 1 election: Requested vote from peers fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277), 83f4c764308448059050d97df4271080 (127.30.255.130:35291)
I20260812 11:16:39.692842 13742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "84717fdae41747658340f4b452e62248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd06182d8bc84772926e4aecaa712379"
I20260812 11:16:39.692941 13742 raft_consensus.cc:3037] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:39.693969 13742 raft_consensus.cc:2454] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84717fdae41747658340f4b452e62248 in term 1.
I20260812 11:16:39.694394 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "84717fdae41747658340f4b452e62248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f4c764308448059050d97df4271080"
I20260812 11:16:39.694442 13083 leader_election.cc:302] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84717fdae41747658340f4b452e62248, fd06182d8bc84772926e4aecaa712379; no voters:
I20260812 11:16:39.694517 13491 raft_consensus.cc:3037] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:39.694602 13888 raft_consensus.cc:2781] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:39.694723 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.690666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:33791 (local address 127.30.255.129:36147)
0812 11:16:39.690860 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.690864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.691548 (+ 684us) server_negotiation.cc:407] Connection header received
0812 11:16:39.691664 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.691667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.691704 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.691761 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.692036 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.692509 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.693607 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.694154 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.694268 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.694315 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.694538 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.694626 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 11:16:39.694747 13888 raft_consensus.cc:686] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 1 LEADER]: Becoming Leader. State: Replica: 84717fdae41747658340f4b452e62248, State: Running, Role: LEADER
I20260812 11:16:39.695032 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.690616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36147 (local address 127.30.255.131:33791)
0812 11:16:39.691479 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.691491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.691579 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.691842 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.691845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.691859 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.691938 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.691944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.692622 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.692626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.693479 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.693489 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.694257 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.694260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.694378 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.694407 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.694835 (+ 428us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.694913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":73,"threads_started":1}
I20260812 11:16:39.695047 13888 consensus_queue.cc:237] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } }
I20260812 11:16:39.695699 13491 raft_consensus.cc:2454] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84717fdae41747658340f4b452e62248 in term 1.
I20260812 11:16:39.696604 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.691427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:48403 (local address 127.30.255.130:35291)
0812 11:16:39.691547 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.691551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.691565 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:39.691604 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.691606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.691645 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.691715 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:39.691928 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.692544 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.695579 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.696112 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.696218 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.696258 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.696460 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.696514 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:39.696655 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.690570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:35291 (local address 127.30.255.131:48403)
0812 11:16:39.690971 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:39.690984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:39.691055 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:39.691722 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:39.691725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:39.691740 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:39.691829 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.691835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.692650 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.692654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.695442 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:39.695453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.696235 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.696239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:39.696335 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.696359 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:16:39.696434 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:16:39.696497 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":74,"threads_started":1}
I20260812 11:16:39.697075 13208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "fd06182d8bc84772926e4aecaa712379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84717fdae41747658340f4b452e62248" is_pre_election: true
I20260812 11:16:39.697145 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "181f6a28bb284e6ab254bb4983a59aeb" candidate_uuid: "fd06182d8bc84772926e4aecaa712379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f4c764308448059050d97df4271080" is_pre_election: true
I20260812 11:16:39.697330 13491 raft_consensus.cc:2379] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd06182d8bc84772926e4aecaa712379 in current term 1: Already voted for candidate 84717fdae41747658340f4b452e62248 in this term.
I20260812 11:16:39.698125 13621 leader_election.cc:302] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd06182d8bc84772926e4aecaa712379; no voters: 83f4c764308448059050d97df4271080, 84717fdae41747658340f4b452e62248
I20260812 11:16:39.698349 13890 raft_consensus.cc:2726] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:16:39.699348 11480 catalog_manager.cc:5495] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84717fdae41747658340f4b452e62248 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "84717fdae41747658340f4b452e62248" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:39.762531 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.758550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.30.255.129:36147)
0812 11:16:39.758699 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.758704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.758819 (+ 115us) server_negotiation.cc:407] Connection header received
0812 11:16:39.758972 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.758975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.759027 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.759110 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:39.759372 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.759864 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.760961 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.761177 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.761255 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.761475 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:39.761840 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:39.762328 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.762359 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.762407 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:16:39.767910 13491 raft_consensus.cc:1261] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Refusing update from remote peer 84717fdae41747658340f4b452e62248: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:39.767957 13742 raft_consensus.cc:1261] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Refusing update from remote peer 84717fdae41747658340f4b452e62248: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:39.768525 13899 consensus_queue.cc:1035] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:39.768709 13888 consensus_queue.cc:1035] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:39.775931 13913 mvcc.cc:205] Tried to move back new op lower bound from 7317640805442322432 to 7317640805153665024. Current Snapshot: MvccSnapshot[applied={T|T < 7317640805442322432}]
I20260812 11:16:39.779217 13911 mvcc.cc:205] Tried to move back new op lower bound from 7317640805442322432 to 7317640805153665024. Current Snapshot: MvccSnapshot[applied={T|T < 7317640805442322432}]
I20260812 11:16:39.780189 13914 mvcc.cc:205] Tried to move back new op lower bound from 7317640805442322432 to 7317640805153665024. Current Snapshot: MvccSnapshot[applied={T|T < 7317640805442322432}]
I20260812 11:16:39.787417 11478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:39.787582 11478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:39.791242 11478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7994d6ab0efa484886eb818b2eb0008d: Sending DeleteTablet for 3 replicas of tablet 181f6a28bb284e6ab254bb4983a59aeb
I20260812 11:16:39.792647 13698 tablet_service.cc:1514] Processing DeleteTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:39 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:55260
I20260812 11:16:39.792779 13173 tablet_service.cc:1514] Processing DeleteTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:39 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47236
I20260812 11:16:39.792784 13443 tablet_service.cc:1514] Processing DeleteTablet for tablet 181f6a28bb284e6ab254bb4983a59aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:39 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:49476
I20260812 11:16:39.793195 13930 tablet_replica.cc:330] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: stopping tablet replica
I20260812 11:16:39.793478 13930 raft_consensus.cc:2229] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:39.793852 13930 raft_consensus.cc:2258] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:39.793972 13931 tablet_replica.cc:330] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: stopping tablet replica
I20260812 11:16:39.794226 13931 raft_consensus.cc:2229] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:39.794513 13931 raft_consensus.cc:2258] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:39.794853 13928 tablet_replica.cc:330] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: stopping tablet replica
I20260812 11:16:39.795106 13928 raft_consensus.cc:2229] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:39.795395 13928 raft_consensus.cc:2258] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:39.795800 13931 ts_tablet_manager.cc:1905] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:39.796185 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11422
I20260812 11:16:39.806042 13928 ts_tablet_manager.cc:1905] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:39.808779 13928 ts_tablet_manager.cc:1918] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:39.808859 13928 log.cc:1196] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/181f6a28bb284e6ab254bb4983a59aeb
I20260812 11:16:39.809101 13928 ts_tablet_manager.cc:1939] T 181f6a28bb284e6ab254bb4983a59aeb P fd06182d8bc84772926e4aecaa712379: Deleting consensus metadata
I20260812 11:16:39.811775 13930 ts_tablet_manager.cc:1905] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:39.814754 13930 ts_tablet_manager.cc:1918] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:39.814846 13930 log.cc:1196] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/181f6a28bb284e6ab254bb4983a59aeb
I20260812 11:16:39.815176 13930 ts_tablet_manager.cc:1939] T 181f6a28bb284e6ab254bb4983a59aeb P 84717fdae41747658340f4b452e62248: Deleting consensus metadata
I20260812 11:16:39.815867 13931 ts_tablet_manager.cc:1918] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:39.815964 13931 log.cc:1196] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/181f6a28bb284e6ab254bb4983a59aeb
I20260812 11:16:39.816322 13931 ts_tablet_manager.cc:1939] T 181f6a28bb284e6ab254bb4983a59aeb P 83f4c764308448059050d97df4271080: Deleting consensus metadata
I20260812 11:16:39.820469 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.795543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.30.255.189:38361)
0812 11:16:39.811760 (+ 16217us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.811765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.811782 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:16:39.811825 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.811828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.811881 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.812849 (+ 968us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.812968 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.813754 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.814919 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.815256 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.817064 (+ 1808us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.817188 (+ 124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.817191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.817194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.817212 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.818861 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.819295 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.819298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.819300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.819388 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.819645 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.819648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.819650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.819827 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.819940 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.820182 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.820304 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16138,"thread_start_us":66,"threads_started":1}
W20260812 11:16:39.821470 13083 connection.cc:535] server connection from 127.0.0.1:47236 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 11:16:39.821930 13356 connection.cc:535] server connection from 127.0.0.1:49476 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 11:16:39.822228 13621 connection.cc:535] server connection from 127.0.0.1:55260 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:16:39.828163 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.795141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.30.255.188:44681)
0812 11:16:39.817696 (+ 22555us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.817702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.820623 (+ 2921us) server_negotiation.cc:407] Connection header received
0812 11:16:39.820678 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.820681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.820734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.821501 (+ 767us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.821597 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.822275 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.823325 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.823624 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.823722 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.824997 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.825001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.825004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.825029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.826603 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.826993 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.826997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.826999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.827078 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.827351 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.827354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.827356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.827531 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.827648 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.827878 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.827993 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22471,"thread_start_us":89,"threads_started":1}
I20260812 11:16:39.840183 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.831613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.30.255.189:38361)
0812 11:16:39.831726 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.831730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.831743 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:39.832037 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.832040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.832088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.832156 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.832965 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.833467 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.834542 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.834841 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.834901 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.837042 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.837046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.837049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.837073 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.838412 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.838971 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.838974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.838976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.839055 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.839340 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.839344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.839345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.839564 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.839823 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.839918 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.840035 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:39.841150 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:39.831805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.30.255.188:44681)
0812 11:16:39.831925 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:39.831930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:39.832137 (+ 207us) server_negotiation.cc:407] Connection header received
0812 11:16:39.832278 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:39.832281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:39.832321 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:39.832382 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:39.833036 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.833802 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.834974 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:39.835416 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:39.835511 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:39.836929 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:39.836933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:39.836936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:39.836961 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:39.838474 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.838940 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.838944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.838945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.839028 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:39.840330 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:39.840333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:39.840335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:39.840544 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:39.840665 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:39.840893 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 11:16:39.841008 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:40.779656 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:40.779171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:32771)
0812 11:16:40.779417 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:40.779532 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
W20260812 11:16:40.780009 13818 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:38763 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:40.781358 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:40.780860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:34169)
0812 11:16:40.781182 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:40.781268 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":67,"threads_started":1}
W20260812 11:16:40.781641 13313 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:38763 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:40.783599 13948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:40.783160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:50103)
0812 11:16:40.783408 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:40.783496 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
W20260812 11:16:40.783865 13570 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:38763 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.291980 13950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7994d6ab0efa484886eb818b2eb0008d)
I20260812 11:16:41.292127 13950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } }
I20260812 11:16:41.292659 13950 leader_election.cc:288] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763), 353be5983f544708a027d37acbe885b4 (127.30.255.189:38361)
I20260812 11:16:41.293109 13951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7994d6ab0efa484886eb818b2eb0008d)
I20260812 11:16:41.293228 13952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.292770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47890)
0812 11:16:41.293035 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.293124 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":61,"threads_started":1}
I20260812 11:16:41.293437 13951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } }
W20260812 11:16:41.293596 12925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.294441 13952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.294213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47904)
0812 11:16:41.294321 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.294375 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 11:16:41.294600 12925 leader_election.cc:334] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763): Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.295176 13951 leader_election.cc:288] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763), 21adb03acb064157b5540806312e2b84 (127.30.255.188:44681)
I20260812 11:16:41.295653 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.294878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47912)
0812 11:16:41.295469 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.295540 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":70,"threads_started":1}
W20260812 11:16:41.295908 12595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.296926 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.296662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47926)
0812 11:16:41.296809 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.296857 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260812 11:16:41.297096 12595 leader_election.cc:334] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763): Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.297457 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.292979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45274 (local address 127.30.255.189:38361)
0812 11:16:41.293285 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.293290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.293483 (+ 193us) server_negotiation.cc:407] Connection header received
0812 11:16:41.293535 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.293538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.293580 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.293656 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.294040 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.294801 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.296203 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.296824 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.296909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.296960 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.297272 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.297346 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":107,"threads_started":1}
I20260812 11:16:41.297853 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.292877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:38361 (local address 127.0.0.1:45274)
0812 11:16:41.293287 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.293300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.293404 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.293779 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.293783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.293803 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.293914 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.293922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.294916 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.294919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.296075 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.296083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.297048 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.297051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.297142 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.297165 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.297649 (+ 484us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.297732 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":66,"threads_started":1}
I20260812 11:16:41.298242 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21adb03acb064157b5540806312e2b84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "353be5983f544708a027d37acbe885b4" is_pre_election: true
I20260812 11:16:41.298377 12635 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21adb03acb064157b5540806312e2b84 in term 1.
I20260812 11:16:41.298729 12924 leader_election.cc:302] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21adb03acb064157b5540806312e2b84, 353be5983f544708a027d37acbe885b4; no voters: 7994d6ab0efa484886eb818b2eb0008d
I20260812 11:16:41.298972 13950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 11:16:41.299032 13950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7994d6ab0efa484886eb818b2eb0008d)
I20260812 11:16:41.299078 13950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:16:41.299597 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.295364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:44681 (local address 127.0.0.1:34214)
0812 11:16:41.295528 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.295540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.295623 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.296246 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.296249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.296266 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.296341 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.296346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.297263 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.297265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.298345 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.298357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.299213 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.299216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.299322 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.299345 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.299411 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.299475 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 11:16:41.299629 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.295725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.30.255.188:44681)
0812 11:16:41.295954 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.295962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.295974 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:41.296012 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.296015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.296068 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.296151 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.296451 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.297148 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.298477 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.298996 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.299114 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.299159 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.299438 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.299508 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260812 11:16:41.300009 12970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "353be5983f544708a027d37acbe885b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "21adb03acb064157b5540806312e2b84" is_pre_election: true
I20260812 11:16:41.300518 13950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } }
I20260812 11:16:41.300822 12970 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 353be5983f544708a027d37acbe885b4 in current term 2: Already voted for candidate 21adb03acb064157b5540806312e2b84 in this term.
I20260812 11:16:41.300941 13950 leader_election.cc:288] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 election: Requested vote from peers 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763), 353be5983f544708a027d37acbe885b4 (127.30.255.189:38361)
I20260812 11:16:41.301230 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.301040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47938)
0812 11:16:41.301131 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.301178 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 11:16:41.301510 12594 leader_election.cc:302] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 353be5983f544708a027d37acbe885b4; no voters: 21adb03acb064157b5540806312e2b84, 7994d6ab0efa484886eb818b2eb0008d
I20260812 11:16:41.301571 12635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21adb03acb064157b5540806312e2b84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "353be5983f544708a027d37acbe885b4"
I20260812 11:16:41.301672 12635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:16:41.302645 12635 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21adb03acb064157b5540806312e2b84 in term 2.
I20260812 11:16:41.302750 13951 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:16:41.302999 12924 leader_election.cc:302] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21adb03acb064157b5540806312e2b84, 353be5983f544708a027d37acbe885b4; no voters:
I20260812 11:16:41.303165 13950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:16:41.303205 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.302941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47950)
0812 11:16:41.303089 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.303138 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 11:16:41.303292 12925 leader_election.cc:334] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763): Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
I20260812 11:16:41.303388 13950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [term 2 LEADER]: Becoming Leader. State: Replica: 21adb03acb064157b5540806312e2b84, State: Running, Role: LEADER
I20260812 11:16:41.303639 13950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } }
I20260812 11:16:41.305300 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21adb03acb064157b5540806312e2b84. Latest consensus state: current_term: 2 leader_uuid: "21adb03acb064157b5540806312e2b84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:41.305436 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:41.305748 13967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:41.307938 13967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:41.308317 13967 catalog_manager.cc:1259] Loaded cluster ID: a1fafdfbec5649a498ce79bff06324e7
I20260812 11:16:41.308365 13967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:41.308866 13967 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:41.309299 13967 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Loaded TSK: 0
I20260812 11:16:41.309929 13967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:16:41.344601 12946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:41.348868 12946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:41.351985 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.351729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47964)
0812 11:16:41.351837 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.351898 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 11:16:41.352041 12635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [term 2 FOLLOWER]: Refusing update from remote peer 21adb03acb064157b5540806312e2b84: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 11:16:41.352526 13950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 11:16:41.352651 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.352390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.0.0.1:47978)
0812 11:16:41.352501 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.352554 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 11:16:41.352826 12925 consensus_peers.cc:487] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 -> Peer 7994d6ab0efa484886eb818b2eb0008d (127.30.255.190:38763): Couldn't send request to peer 7994d6ab0efa484886eb818b2eb0008d. Status: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 11:16:41.354122 13951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21adb03acb064157b5540806312e2b84. Latest consensus state: current_term: 2 leader_uuid: "21adb03acb064157b5540806312e2b84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:41.354256 13951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:41.355434 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "21adb03acb064157b5540806312e2b84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:41.355569 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:41.356105 13969 mvcc.cc:205] Tried to move back new op lower bound from 7317640811933065216 to 7317640811742179328. Current Snapshot: MvccSnapshot[applied={T|T < 7317640811933065216}]
I20260812 11:16:41.356235 13951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "21adb03acb064157b5540806312e2b84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7994d6ab0efa484886eb818b2eb0008d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 38763 } } peers { permanent_uuid: "353be5983f544708a027d37acbe885b4" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 38361 } attrs { promote: false } } peers { permanent_uuid: "21adb03acb064157b5540806312e2b84" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 44681 } attrs { promote: false } } }
I20260812 11:16:41.356341 13951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353be5983f544708a027d37acbe885b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:16:41.367851 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.361895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:35291 (local address 127.0.0.1:58518)
0812 11:16:41.362043 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.362058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.362148 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.362905 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.362909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.362932 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.363046 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.363054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.364286 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.364290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.365511 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.365524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.367404 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.367408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.367522 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.367550 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.367625 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.367702 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 11:16:41.368252 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.362411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.30.255.130:35291)
0812 11:16:41.362651 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.362655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.362671 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:16:41.362723 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.362725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.362777 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.362871 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.363425 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.364171 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.365676 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.367132 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.367256 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.367338 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.368018 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.368109 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":83,"threads_started":1}
I20260812 11:16:41.368522 13952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.362973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39277 (local address 127.0.0.1:58128)
0812 11:16:41.363230 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.363243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.363339 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.364636 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.364639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.364656 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.364742 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.364748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.365804 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.365808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.367160 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.367171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.368172 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.368175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.368262 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.368285 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.368332 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.368408 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":78}
I20260812 11:16:41.368826 13443 tablet_service.cc:1467] Processing CreateTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed (DEFAULT_TABLE table=test-table [id=841ca1f3c4c94131856cd7ac4afea75f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:41.368992 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.363301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36147 (local address 127.0.0.1:42938)
0812 11:16:41.364235 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.364248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.364335 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.364712 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.364714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.364728 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.364794 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.364799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.365804 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.365808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.367160 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.367171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.368628 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.368632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.368718 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.368741 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.368804 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.368874 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":105,"thread_start_us":85,"threads_started":1}
I20260812 11:16:41.369171 13443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02acfb10f29b42dbb47dbe4247b316ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:41.369266 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.363223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.30.255.131:39277)
0812 11:16:41.364410 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.364414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.364427 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:41.364473 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.364476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.364518 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.364845 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.364954 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.365690 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.367326 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.368030 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.368957 (+ 927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.369011 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.369048 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.369140 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":84,"threads_started":1}
I20260812 11:16:41.369480 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.363762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.30.255.129:36147)
0812 11:16:41.363997 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.364001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.364410 (+ 409us) server_negotiation.cc:407] Connection header received
0812 11:16:41.364471 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.364473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.364519 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.364594 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.364900 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.365673 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.367342 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.368178 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.368275 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.369236 (+ 961us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.369287 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.369363 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260812 11:16:41.369712 13698 tablet_service.cc:1467] Processing CreateTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed (DEFAULT_TABLE table=test-table [id=841ca1f3c4c94131856cd7ac4afea75f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:41.369958 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02acfb10f29b42dbb47dbe4247b316ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:41.371629 13173 tablet_service.cc:1467] Processing CreateTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed (DEFAULT_TABLE table=test-table [id=841ca1f3c4c94131856cd7ac4afea75f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:41.371850 13173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02acfb10f29b42dbb47dbe4247b316ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:41.372066 13980 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Bootstrap starting.
I20260812 11:16:41.372990 13980 tablet_bootstrap.cc:654] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:41.374030 13983 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Bootstrap starting.
I20260812 11:16:41.374243 13980 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: No bootstrap required, opened a new log
I20260812 11:16:41.374375 13980 ts_tablet_manager.cc:1397] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:41.374500 13984 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: Bootstrap starting.
I20260812 11:16:41.374806 13980 raft_consensus.cc:348] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.374898 13980 raft_consensus.cc:374] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:41.374928 13980 raft_consensus.cc:729] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f4c764308448059050d97df4271080, State: Initialized, Role: FOLLOWER
I20260812 11:16:41.374943 13983 tablet_bootstrap.cc:654] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:41.375077 13980 consensus_queue.cc:260] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.375381 13984 tablet_bootstrap.cc:654] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:41.375429 13980 ts_tablet_manager.cc:1428] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:41.376679 13984 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: No bootstrap required, opened a new log
I20260812 11:16:41.376770 13984 ts_tablet_manager.cc:1397] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 11:16:41.376904 13983 tablet_bootstrap.cc:492] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: No bootstrap required, opened a new log
I20260812 11:16:41.376992 13983 ts_tablet_manager.cc:1397] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 11:16:41.377195 13984 raft_consensus.cc:348] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.377300 13984 raft_consensus.cc:374] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:41.377331 13984 raft_consensus.cc:729] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84717fdae41747658340f4b452e62248, State: Initialized, Role: FOLLOWER
I20260812 11:16:41.377374 13983 raft_consensus.cc:348] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.377476 13983 raft_consensus.cc:374] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:41.377439 13984 consensus_queue.cc:260] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.377506 13983 raft_consensus.cc:729] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd06182d8bc84772926e4aecaa712379, State: Initialized, Role: FOLLOWER
I20260812 11:16:41.377650 13983 consensus_queue.cc:260] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.377748 13984 ts_tablet_manager.cc:1428] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:41.377972 13983 ts_tablet_manager.cc:1428] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:41.378005 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.375841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:55927)
0812 11:16:41.377863 (+ 2022us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.377919 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1959,"thread_start_us":95,"threads_started":1}
I20260812 11:16:41.378774 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.378378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:48419)
0812 11:16:41.378612 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.378667 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":105,"threads_started":1}
I20260812 11:16:41.379136 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.378591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:56255)
0812 11:16:41.378982 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.379041 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260812 11:16:41.380597 13569 heartbeater.cc:503] Master 127.30.255.188:44681 was elected leader, sending a full tablet report...
I20260812 11:16:41.381148 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.378956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:41495)
0812 11:16:41.381006 (+ 2050us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.381058 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1974,"thread_start_us":85,"threads_started":1}
I20260812 11:16:41.381392 13311 heartbeater.cc:503] Master 127.30.255.188:44681 was elected leader, sending a full tablet report...
I20260812 11:16:41.382056 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.381565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:47729)
0812 11:16:41.381964 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.382007 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330}
I20260812 11:16:41.382521 13813 heartbeater.cc:503] Master 127.30.255.188:44681 was elected leader, sending a full tablet report...
I20260812 11:16:41.382838 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.382640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:43991)
0812 11:16:41.382727 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.382774 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 11:16:41.619611 13985 raft_consensus.cc:482] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:41.619753 13985 raft_consensus.cc:504] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.620875 13985 leader_election.cc:288] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84717fdae41747658340f4b452e62248 (127.30.255.129:36147), fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:41.624794 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.621212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:48765 (local address 127.30.255.131:39277)
0812 11:16:41.621332 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.621336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.621439 (+ 103us) server_negotiation.cc:407] Connection header received
0812 11:16:41.621572 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.621574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.621615 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.621679 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.621975 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.622533 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.623757 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.624176 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.624291 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.624364 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.624590 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.624662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:16:41.624795 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.621200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37587 (local address 127.30.255.129:36147)
0812 11:16:41.621330 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.621335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.621401 (+ 66us) server_negotiation.cc:407] Connection header received
0812 11:16:41.621556 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.621559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.621599 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.621662 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:41.621984 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.622511 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.623751 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.624150 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.624268 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.624323 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.624603 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.624673 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:41.624799 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.621096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39277 (local address 127.30.255.130:48765)
0812 11:16:41.621351 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.621364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.621471 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.621697 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.621700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.621715 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.621853 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.621859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.622634 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.622638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.623643 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.623653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.624311 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.624314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.624467 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.624493 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.624561 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.624641 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":111}
I20260812 11:16:41.624846 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.621096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36147 (local address 127.30.255.130:37587)
0812 11:16:41.621328 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.621342 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:41.621455 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:41.621682 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:41.621684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:41.621700 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:41.621856 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.621862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.622614 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.622624 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.623638 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:41.623649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.624293 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.624297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:41.624458 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.624484 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:16:41.624550 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 11:16:41.624669 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":89,"spinlock_wait_cycles":1792}
I20260812 11:16:41.625252 13208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02acfb10f29b42dbb47dbe4247b316ed" candidate_uuid: "83f4c764308448059050d97df4271080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84717fdae41747658340f4b452e62248" is_pre_election: true
I20260812 11:16:41.625391 13742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02acfb10f29b42dbb47dbe4247b316ed" candidate_uuid: "83f4c764308448059050d97df4271080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd06182d8bc84772926e4aecaa712379" is_pre_election: true
I20260812 11:16:41.625452 13208 raft_consensus.cc:2454] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f4c764308448059050d97df4271080 in term 0.
I20260812 11:16:41.625514 13742 raft_consensus.cc:2454] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f4c764308448059050d97df4271080 in term 0.
I20260812 11:16:41.625887 13354 leader_election.cc:302] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f4c764308448059050d97df4271080, fd06182d8bc84772926e4aecaa712379; no voters:
I20260812 11:16:41.626116 13985 raft_consensus.cc:2781] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:41.626173 13985 raft_consensus.cc:482] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:41.626211 13985 raft_consensus.cc:3037] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:41.627327 13985 raft_consensus.cc:504] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.627655 13985 leader_election.cc:288] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [CANDIDATE]: Term 1 election: Requested vote from peers 84717fdae41747658340f4b452e62248 (127.30.255.129:36147), fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277)
I20260812 11:16:41.627967 13742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02acfb10f29b42dbb47dbe4247b316ed" candidate_uuid: "83f4c764308448059050d97df4271080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd06182d8bc84772926e4aecaa712379"
I20260812 11:16:41.627967 13208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02acfb10f29b42dbb47dbe4247b316ed" candidate_uuid: "83f4c764308448059050d97df4271080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84717fdae41747658340f4b452e62248"
I20260812 11:16:41.628085 13208 raft_consensus.cc:3037] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:41.628085 13742 raft_consensus.cc:3037] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:41.629166 13742 raft_consensus.cc:2454] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f4c764308448059050d97df4271080 in term 1.
I20260812 11:16:41.629207 13208 raft_consensus.cc:2454] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f4c764308448059050d97df4271080 in term 1.
I20260812 11:16:41.629441 13354 leader_election.cc:302] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 83f4c764308448059050d97df4271080, 84717fdae41747658340f4b452e62248, fd06182d8bc84772926e4aecaa712379; no voters:
I20260812 11:16:41.629614 13985 raft_consensus.cc:2781] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:41.629851 13985 raft_consensus.cc:686] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 1 LEADER]: Becoming Leader. State: Replica: 83f4c764308448059050d97df4271080, State: Running, Role: LEADER
I20260812 11:16:41.630313 13985 consensus_queue.cc:237] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } }
I20260812 11:16:41.630519 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.630249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:46195)
0812 11:16:41.630391 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.630444 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 11:16:41.633028 12951 catalog_manager.cc:5495] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f4c764308448059050d97df4271080 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "83f4c764308448059050d97df4271080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f4c764308448059050d97df4271080" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 35291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:41.660130 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.657165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.30.255.130:35291)
0812 11:16:41.657282 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:41.657285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:41.657297 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:41.657413 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:41.657415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:41.657463 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:41.657545 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:41.657715 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.658199 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.658964 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:41.659156 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:41.659219 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:41.659427 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:41.659841 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:41.659925 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:41.659972 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 11:16:41.660015 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:41.662514 13742 raft_consensus.cc:1261] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Refusing update from remote peer 83f4c764308448059050d97df4271080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:41.662673 13208 raft_consensus.cc:1261] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248 [term 1 FOLLOWER]: Refusing update from remote peer 83f4c764308448059050d97df4271080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:41.662988 13985 consensus_queue.cc:1035] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd06182d8bc84772926e4aecaa712379" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:41.663170 14007 consensus_queue.cc:1035] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84717fdae41747658340f4b452e62248" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:41.664680 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.664425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:47045)
0812 11:16:41.664543 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.664604 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 11:16:41.665717 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.665447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:35701)
0812 11:16:41.665599 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.665651 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 11:16:41.669188 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.668888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.131:35801)
0812 11:16:41.669070 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.669123 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 11:16:41.669191 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.668954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.129:48575)
0812 11:16:41.669077 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.669130 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:16:41.671197 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.670985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:42071)
0812 11:16:41.671095 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.671142 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 11:16:41.671972 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.671636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:38763 (local address 127.30.255.130:60277)
0812 11:16:41.671853 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.671911 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260812 11:16:41.677500 12946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:41.677623 12946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:41.681087 12946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21adb03acb064157b5540806312e2b84: Sending DeleteTablet for 3 replicas of tablet 02acfb10f29b42dbb47dbe4247b316ed
I20260812 11:16:41.681782 13443 tablet_service.cc:1514] Processing DeleteTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:41 UTC) from {username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:58518
I20260812 11:16:41.681952 13698 tablet_service.cc:1514] Processing DeleteTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:41 UTC) from {username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:58128
I20260812 11:16:41.682206 13173 tablet_service.cc:1514] Processing DeleteTablet for tablet 02acfb10f29b42dbb47dbe4247b316ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:41 UTC) from {username='oryx', principal='oryx/127.30.255.188@KRBTEST.COM'} at 127.0.0.1:42938
I20260812 11:16:41.682237 14036 tablet_replica.cc:330] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: stopping tablet replica
I20260812 11:16:41.682353 14036 raft_consensus.cc:2229] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:41.682510 14038 tablet_replica.cc:330] T 02acfb10f29b42dbb47dbe4247b316ed P 84717fdae41747658340f4b452e62248: stopping tablet replica
I20260812 11:16:41.682693 14036 raft_consensus.cc:2258] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:41.682814 14037 tablet_replica.cc:330] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: stopping tablet replica
I20260812 11:16:41.682960 14037 raft_consensus.cc:2229] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:41.683218 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 12890
I20260812 11:16:41.683312 14037 raft_consensus.cc:2258] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:41.686702 14036 ts_tablet_manager.cc:1905] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:41.687201 14037 ts_tablet_manager.cc:1905] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:41.689286 14036 ts_tablet_manager.cc:1918] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:41.689354 14036 log.cc:1196] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/02acfb10f29b42dbb47dbe4247b316ed
I20260812 11:16:41.689618 14036 ts_tablet_manager.cc:1939] T 02acfb10f29b42dbb47dbe4247b316ed P 83f4c764308448059050d97df4271080: Deleting consensus metadata
I20260812 11:16:41.689677 14037 ts_tablet_manager.cc:1918] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:41.689738 14037 log.cc:1196] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/02acfb10f29b42dbb47dbe4247b316ed
I20260812 11:16:41.689962 14037 ts_tablet_manager.cc:1939] T 02acfb10f29b42dbb47dbe4247b316ed P fd06182d8bc84772926e4aecaa712379: Deleting consensus metadata
I20260812 11:16:41.690289 12923 catalog_manager.cc:4841] TS 83f4c764308448059050d97df4271080 (127.30.255.130:35291): tablet 02acfb10f29b42dbb47dbe4247b316ed (table test-table [id=841ca1f3c4c94131856cd7ac4afea75f]) successfully deleted
I20260812 11:16:41.690495 12923 catalog_manager.cc:4841] TS fd06182d8bc84772926e4aecaa712379 (127.30.255.131:39277): tablet 02acfb10f29b42dbb47dbe4247b316ed (table test-table [id=841ca1f3c4c94131856cd7ac4afea75f]) successfully deleted
I20260812 11:16:41.730968 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 13326
I20260812 11:16:41.733565 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:41.730998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36147 (local address 127.0.0.1:42954)
0812 11:16:41.731164 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:41.731227 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 11:16:41.735810 12926 catalog_manager.cc:4584] TS 84717fdae41747658340f4b452e62248 (127.30.255.129:36147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02acfb10f29b42dbb47dbe4247b316ed: Network error: Client connection negotiation failed: client connection to 127.30.255.129:36147: connect: Connection refused (error 111)
I20260812 11:16:41.740590 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 13584
I20260812 11:16:41.753247 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 12362
I20260812 11:16:41.761452 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 12569
2026-08-12T11:16:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6066 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:41 dist-test-slave-gjrz krb5kdc[14042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:41 dist-test-slave-gjrz krb5kdc[14042](info): set up 2 sockets
Aug 12 11:16:41 dist-test-slave-gjrz krb5kdc[14042](info): commencing operation
krb5kdc: starting...
W20260812 11:16:42.967176 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab.
Entry for principal oryx/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab.
I20260812 11:16:43.083523 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43543
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab
--principal=oryx/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:43.189467 14062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:43.189687 14062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:43.189740 14062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:43.193611 14062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:43.193660 14062 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:16:43.193682 14062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:43.193706 14062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:43.193734 14062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:43.193756 14062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:43.198668 14062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43543
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.190.keytab
--principal=oryx/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:43.199566 14062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:43.200448 14062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:43.208348 14074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:43.208359 14079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:43.208354 14075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.208503 14062 server_base.cc:1034] running on GCE node
I20260812 11:16:43.209095 14062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:43.210011 14062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:43.211167 14062 hybrid_clock.cc:648] HybridClock initialized: now 1786533403211152 us; error 25 us; skew 500 ppm
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:43.213627 14062 init.cc:377] Logged in from keytab as oryx/127.30.255.190@KRBTEST.COM (short username oryx)
I20260812 11:16:43.214450 14062 webserver.cc:460] Webserver started at http://127.30.255.190:33735/ using document root <none> and password file <none>
I20260812 11:16:43.214933 14062 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:43.214993 14062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:43.215193 14062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:43.217077 14062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "3002a2fb104942daaadc5a5d5105f6e5"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "91dbcbd9e40db65d7a550017778cd418"
server_key_iv: "74183a64d0e09575472506230d80bf39"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.217587 14062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "3002a2fb104942daaadc5a5d5105f6e5"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "91dbcbd9e40db65d7a550017778cd418"
server_key_iv: "74183a64d0e09575472506230d80bf39"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.221092 14062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:43.223219 14094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.224406 14062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:43.224525 14062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "3002a2fb104942daaadc5a5d5105f6e5"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "91dbcbd9e40db65d7a550017778cd418"
server_key_iv: "74183a64d0e09575472506230d80bf39"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.224614 14062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:43.248045 14062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:43.250536 14062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:43.250698 14062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:43.257999 14062 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:43543
I20260812 11:16:43.258008 14212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:43543 every 8 connection(s)
I20260812 11:16:43.258857 14062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:43.261463 14213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:43.266636 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Bootstrap starting.
I20260812 11:16:43.268451 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 14062
I20260812 11:16:43.268584 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:43.268885 31742 external_mini_cluster.cc:1428] Setting key bbf1e1f3ce279c77507f2a3d5da6fe32
I20260812 11:16:43.269271 14213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:43.270155 14213 log.cc:824] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:43.272087 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: No bootstrap required, opened a new log
I20260812 11:16:43.276068 14213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } }
I20260812 11:16:43.276368 14213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:43.276440 14213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002a2fb104942daaadc5a5d5105f6e5, State: Initialized, Role: FOLLOWER
I20260812 11:16:43.276932 14213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } }
I20260812 11:16:43.277096 14213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:43.277184 14213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:43.277287 14213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:43.278127 14213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } }
I20260812 11:16:43.278491 14213 leader_election.cc:302] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3002a2fb104942daaadc5a5d5105f6e5; no voters:
I20260812 11:16:43.278821 14213 leader_election.cc:288] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:43.278893 14222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:43.279093 14222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3002a2fb104942daaadc5a5d5105f6e5, State: Running, Role: LEADER
I20260812 11:16:43.279417 14222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } }
I20260812 11:16:43.280064 14213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:43.280534 14229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3002a2fb104942daaadc5a5d5105f6e5. Latest consensus state: current_term: 1 leader_uuid: "3002a2fb104942daaadc5a5d5105f6e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } } }
I20260812 11:16:43.281623 14229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:43.280706 14228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3002a2fb104942daaadc5a5d5105f6e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3002a2fb104942daaadc5a5d5105f6e5" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43543 } } }
I20260812 11:16:43.282064 14228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5 [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:43.282394 14250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:43.283406 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.270023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.30.255.190:43543)
0812 11:16:43.270397 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.270412 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.270448 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:16:43.271214 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.271234 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.271588 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.271853 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.272735 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.273718 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.274900 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.275418 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.275528 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.278226 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.278233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.278249 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.278278 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.279961 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.280418 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.280424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.280430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.280588 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.280966 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.280969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.280971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.281385 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.281566 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.282327 (+ 761us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.282603 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260812 11:16:43.285252 14250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:43.290337 14250 catalog_manager.cc:1347] Generated new cluster ID: 28b0dee4a4ef45678ca12753c1a076ba
I20260812 11:16:43.290403 14250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:43.305371 14250 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:43.306195 14250 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:43.314237 14250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Generated new TSK 0
I20260812 11:16:43.314806 14250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
I20260812 11:16:43.349776 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43543
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:43.455771 14263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:43.455984 14263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:43.456023 14263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:43.459811 14263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:43.459856 14263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:43.459890 14263 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:43.459924 14263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:43.464815 14263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:43.465668 14263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:43.466534 14263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:43.475217 14278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:43.475286 14273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.475332 14263 server_base.cc:1034] running on GCE node
W20260812 11:16:43.475250 14274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.475946 14263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:43.476526 14263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:43.477672 14263 hybrid_clock.cc:648] HybridClock initialized: now 1786533403477650 us; error 34 us; skew 500 ppm
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:43.480206 14263 init.cc:377] Logged in from keytab as oryx/127.30.255.129@KRBTEST.COM (short username oryx)
I20260812 11:16:43.481062 14263 webserver.cc:460] Webserver started at http://127.30.255.129:38373/ using document root <none> and password file <none>
I20260812 11:16:43.481628 14263 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:43.481696 14263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:43.481904 14263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:43.483800 14263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "da91b8e22e5859115c4f96f7dc126312"
server_key_iv: "bbf76a6a4571e32c6596376322903ebe"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.484313 14263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "da91b8e22e5859115c4f96f7dc126312"
server_key_iv: "bbf76a6a4571e32c6596376322903ebe"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.487680 14263 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:43.489826 14292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.490986 14263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:43.491149 14263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "da91b8e22e5859115c4f96f7dc126312"
server_key_iv: "bbf76a6a4571e32c6596376322903ebe"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.491364 14263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:43.517498 14263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:43.521317 14263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:43.521552 14263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:43.524816 14263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:43.524880 14263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.524935 14263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:43.524968 14263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:43.537689 14263 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41011
I20260812 11:16:43.537721 14502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41011 every 8 connection(s)
I20260812 11:16:43.538592 14263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:43.538816 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.525625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:55389 (local address 127.30.255.190:43543)
0812 11:16:43.525754 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.525759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.526888 (+ 1129us) server_negotiation.cc:407] Connection header received
0812 11:16:43.528067 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.528070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.528129 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.528211 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.529473 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.530199 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.531630 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.531920 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.532013 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.535184 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.535188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.535191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.535221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.536846 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.537326 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.537330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.537332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.537426 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.537748 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.537751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.537753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.537954 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.538042 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.538528 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.538657 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:43.539693 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.526212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.129:55389)
0812 11:16:43.526700 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.526759 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.527825 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.528374 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.528381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.528758 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.529323 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.529339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.530343 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.530346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.531480 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.531493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.532082 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.532086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.532263 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.532921 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.532943 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.534957 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.536957 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.536963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.536978 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.537229 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.537507 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.537511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.537513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.537659 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.538057 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.538064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.538304 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.538468 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.538781 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
I20260812 11:16:43.540872 14503 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43543
I20260812 11:16:43.540997 14503 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:43.541461 14503 heartbeater.cc:511] Master 127.30.255.190:43543 requested a full tablet report, sending...
I20260812 11:16:43.543033 14121 ts_manager.cc:194] Registered new tserver with Master: 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011)
I20260812 11:16:43.544413 14121 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:55389
I20260812 11:16:43.545392 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 14263
I20260812 11:16:43.545531 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:43.545866 31742 external_mini_cluster.cc:1428] Setting key f0bb92c80472733b7665bcddf6384938
I20260812 11:16:43.549863 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.542920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.129:36463)
0812 11:16:43.543319 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.543332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.543430 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.543638 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.543640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.543833 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.543949 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.543955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.544656 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.544659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.545309 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.545320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.545888 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.545891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.545981 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.546480 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.546494 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.546897 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.548469 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.548472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.548474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.548711 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.548928 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.548930 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.548932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.549019 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.549368 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.549371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.549465 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.549578 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.549713 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":89,"threads_started":1}
I20260812 11:16:43.549911 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.543000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:36463 (local address 127.30.255.190:43543)
0812 11:16:43.543125 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.543129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.543407 (+ 278us) server_negotiation.cc:407] Connection header received
0812 11:16:43.543505 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.543507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.543546 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.543606 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.544038 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.544535 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.545451 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.545766 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.545867 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.547019 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.547023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.547026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.547048 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.548355 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.548792 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.548794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.548795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.548857 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.549100 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.549102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.549103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.549272 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.549369 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.549642 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.549763 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
I20260812 11:16:43.572005 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43543
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:43.678205 14520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:43.678431 14520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:43.678475 14520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:43.682528 14520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:43.682575 14520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:43.682626 14520 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:43.682677 14520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:43.687633 14520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:43.688479 14520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:43.689358 14520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:43.698136 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:43.698165 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.698225 14520 server_base.cc:1034] running on GCE node
W20260812 11:16:43.698107 14526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.698768 14520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:43.699379 14520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:43.700537 14520 hybrid_clock.cc:648] HybridClock initialized: now 1786533403700519 us; error 28 us; skew 500 ppm
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:43.703091 14520 init.cc:377] Logged in from keytab as oryx/127.30.255.130@KRBTEST.COM (short username oryx)
I20260812 11:16:43.703923 14520 webserver.cc:460] Webserver started at http://127.30.255.130:45585/ using document root <none> and password file <none>
I20260812 11:16:43.704411 14520 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:43.704474 14520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:43.704663 14520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:43.706554 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "687bc46e13ff4fac808cd77d9cb59638"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "15a78765bfc09260ca4e5acd93c1226f"
server_key_iv: "730de49cde07c2670883189a48dd1b6c"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.707130 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "687bc46e13ff4fac808cd77d9cb59638"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "15a78765bfc09260ca4e5acd93c1226f"
server_key_iv: "730de49cde07c2670883189a48dd1b6c"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.710475 14520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:16:43.712693 14541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.713760 14520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:43.713864 14520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "687bc46e13ff4fac808cd77d9cb59638"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "15a78765bfc09260ca4e5acd93c1226f"
server_key_iv: "730de49cde07c2670883189a48dd1b6c"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.713951 14520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:43.731787 14520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:43.734685 14520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:43.734853 14520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:43.736282 14520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:43.736333 14520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.736384 14520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:43.736454 14520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:43.746691 14520 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:33217
I20260812 11:16:43.746843 14771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:33217 every 8 connection(s)
I20260812 11:16:43.747700 14520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:43.749253 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.736907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:51389 (local address 127.30.255.190:43543)
0812 11:16:43.737023 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.737027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.738013 (+ 986us) server_negotiation.cc:407] Connection header received
0812 11:16:43.738906 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.738909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.738966 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.739038 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.740253 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.740973 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.742725 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.743051 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.743161 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.745582 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.745587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.745590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.745622 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.747128 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.747633 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.747636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.747638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.747738 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.748156 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.748159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.748161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.748379 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.748482 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.749038 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.749137 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:16:43.749926 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.737417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.130:51389)
0812 11:16:43.737875 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.737915 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.738712 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.739331 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.739340 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.739663 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.740110 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.740123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.741124 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.741128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.742593 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.742606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.743358 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.743362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.743498 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.743990 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.744006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.745425 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.747235 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.747239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.747250 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.747522 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.747857 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.747860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.747862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.748030 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.748611 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.748616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.748823 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.748963 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.749199 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260812 11:16:43.751000 14772 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43543
I20260812 11:16:43.751132 14772 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:43.751614 14772 heartbeater.cc:511] Master 127.30.255.190:43543 requested a full tablet report, sending...
I20260812 11:16:43.752588 14123 ts_manager.cc:194] Registered new tserver with Master: 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:43.753324 14123 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:51389
I20260812 11:16:43.757000 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 14520
I20260812 11:16:43.757135 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:43.757419 31742 external_mini_cluster.cc:1428] Setting key 3f8dad4f95eab84ae06470e7b9eb0845
I20260812 11:16:43.759135 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.752970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:46911 (local address 127.30.255.190:43543)
0812 11:16:43.753073 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.753076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.753241 (+ 165us) server_negotiation.cc:407] Connection header received
0812 11:16:43.753347 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.753349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.753386 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.753455 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.753878 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.754358 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.755124 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.755383 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.755470 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.756300 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.756304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.756306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.756330 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.757425 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.757811 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.757815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.757816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.757908 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.758188 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.758190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.758191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.758377 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.758462 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.758937 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.759025 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:43.759150 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.752918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.130:46911)
0812 11:16:43.753163 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.753177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.753271 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.753486 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.753489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.753669 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.753767 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.753773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.754473 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.754476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.755012 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.755019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.755479 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.755482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.755568 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.755908 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.755918 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.756175 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.757543 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.757545 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.757546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.757719 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.758008 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.758010 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.758012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.758108 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.758666 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.758669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.758761 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.758865 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.758992 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
I20260812 11:16:43.781955 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:43543
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:43.890410 14792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:43.890625 14792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:43.890664 14792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:43.894567 14792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:43.894624 14792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:43.894672 14792 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:16:43.894722 14792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:43.899997 14792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.255.190:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:43.900825 14792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:43.901716 14792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:43.910665 14802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.910743 14792 server_base.cc:1034] running on GCE node
W20260812 11:16:43.910667 14808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:43.910661 14805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:43.911340 14792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:43.911871 14792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:43.913024 14792 hybrid_clock.cc:648] HybridClock initialized: now 1786533403913009 us; error 31 us; skew 500 ppm
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:43.915539 14792 init.cc:377] Logged in from keytab as oryx/127.30.255.131@KRBTEST.COM (short username oryx)
I20260812 11:16:43.916321 14792 webserver.cc:460] Webserver started at http://127.30.255.131:37605/ using document root <none> and password file <none>
I20260812 11:16:43.916829 14792 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:43.916894 14792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:43.917074 14792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:43.919023 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "e7149eddd07948e2b0303476c6c59e1b"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "1f89ea1ce4fdbe58044d649abfe719f1"
server_key_iv: "bca5a712b04e91c17fe6cd3b7af1126b"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.919571 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "e7149eddd07948e2b0303476c6c59e1b"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "1f89ea1ce4fdbe58044d649abfe719f1"
server_key_iv: "bca5a712b04e91c17fe6cd3b7af1126b"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.923192 14792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:16:43.925408 14822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.926489 14792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:43.926594 14792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "e7149eddd07948e2b0303476c6c59e1b"
format_stamp: "Formatted at 2026-08-12 11:16:43 on dist-test-slave-gjrz"
server_key: "1f89ea1ce4fdbe58044d649abfe719f1"
server_key_iv: "bca5a712b04e91c17fe6cd3b7af1126b"
server_key_version: "encryptionkey@0"
I20260812 11:16:43.926684 14792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:43.942381 14792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:43.944829 14792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:43.944988 14792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:43.946269 14792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:43.946317 14792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:43.946367 14792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:43.946396 14792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:43.956162 14792 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:33589
I20260812 11:16:43.956231 15060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:33589 every 8 connection(s)
I20260812 11:16:43.956960 14792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:43.957492 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 14792
I20260812 11:16:43.957628 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:43.957906 31742 external_mini_cluster.cc:1428] Setting key 35a3c036ced794722e674eb095cd33db
I20260812 11:16:43.959647 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.947083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:54741 (local address 127.30.255.190:43543)
0812 11:16:43.947229 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.947234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.948067 (+ 833us) server_negotiation.cc:407] Connection header received
0812 11:16:43.949034 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.949038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.949101 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.949198 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.950806 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.951540 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.952825 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.953052 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.953147 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.956009 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.956013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.956015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.956041 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.957519 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.958049 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.958053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.958062 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.958158 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.958553 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.958556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.958558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.958754 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.958854 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.959377 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.959499 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:16:43.960332 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.947480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.131:54741)
0812 11:16:43.947909 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.947955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.948798 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.949509 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.949518 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.949960 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.950604 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.950636 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.951680 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.951683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.952703 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.952715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.953207 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.953211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.953375 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.954411 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.954427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.955813 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.957637 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.957643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.957655 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.957956 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.958275 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.958279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.958281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.958439 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.958853 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.958858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.959134 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.959333 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.959566 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
I20260812 11:16:43.961336 15061 heartbeater.cc:347] Connected to a master server at 127.30.255.190:43543
I20260812 11:16:43.961463 15061 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:43.961905 15061 heartbeater.cc:511] Master 127.30.255.190:43543 requested a full tablet report, sending...
I20260812 11:16:43.962818 14121 ts_manager.cc:194] Registered new tserver with Master: e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:43.963523 14121 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:54741
I20260812 11:16:43.969354 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.963350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:56967 (local address 127.30.255.190:43543)
0812 11:16:43.963464 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.963468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.963698 (+ 230us) server_negotiation.cc:407] Connection header received
0812 11:16:43.963851 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.963854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.963892 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.963953 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.964354 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.964835 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.965604 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.965820 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.965903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.966809 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:43.966812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:43.966815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:43.966840 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:43.967952 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.968324 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.968327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.968329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.968402 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:43.968641 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:43.968644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:43.968646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:43.968807 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:43.968892 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:43.969132 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 11:16:43.969226 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:43.969365 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.963237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.30.255.131:56967)
0812 11:16:43.963624 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:43.963638 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:43.963729 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:43.963983 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:43.963986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:43.964158 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:43.964250 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.964255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.964953 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.964955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.965495 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:43.965503 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.965916 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.965920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:43.966009 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.966406 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:43.966417 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:43.966683 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:43.968052 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.968062 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.968064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.968237 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.968482 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:43.968485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:43.968487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:43.968561 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:43.968896 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:43.968900 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:43.968983 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 11:16:43.969076 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 11:16:43.969201 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":99,"threads_started":1}
I20260812 11:16:43.971887 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:16:43.994967 14219 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:43.989637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.30.255.190:43543)
0812 11:16:43.989738 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.989741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.989752 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:16:43.989784 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.989786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.989819 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.989893 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.990490 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.990993 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.991803 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.992030 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.992105 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:43.994843 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42798: BlockingRecv error: recv got EOF from 127.0.0.1:42798 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 11:16:43 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:44.002804 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:43.996286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.30.255.190:43543)
0812 11:16:43.996378 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:43.996381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:43.996500 (+ 119us) server_negotiation.cc:407] Connection header received
0812 11:16:43.996603 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:43.996605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:43.996636 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:43.996694 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:43.997209 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.997666 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.998346 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:43.998531 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:43.998593 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.000501 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:44.000504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:44.000506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:44.000527 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:44.001593 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.001936 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.001939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.001941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.002023 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.002247 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.002249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.002250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.002400 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:44.002489 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.002628 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.002708 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:16:44.005245 14121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:44.008248 14121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:44.021327 15098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.015836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41011 (local address 127.0.0.1:48068)
0812 11:16:44.016136 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.016185 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.016281 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.017585 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.017591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.017616 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.017745 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.017752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.018826 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.018829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.019991 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.020001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.020799 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.020802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.020934 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.020971 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.021079 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.021194 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":23,"thread_start_us":66,"threads_started":1}
I20260812 11:16:44.021337 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.016833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.30.255.129:41011)
0812 11:16:44.017112 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.017125 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.017149 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:16:44.017215 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.017218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.017417 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.017575 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.017885 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.018723 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.020123 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.020681 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.020808 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.020945 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.021112 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.021206 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260812 11:16:44.022356 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.015973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:33217 (local address 127.0.0.1:47704)
0812 11:16:44.016733 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.016746 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.016833 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.017904 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.017907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.017951 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.018135 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.018147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.019553 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.019556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.020865 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.020873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.021912 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.021916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.022018 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.022042 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.022124 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.022228 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":166,"spinlock_wait_cycles":5376,"thread_start_us":76,"threads_started":1}
I20260812 11:16:44.022459 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.016610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47704 (local address 127.30.255.130:33217)
0812 11:16:44.017456 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.017467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.017490 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:16:44.017563 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.017568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.017762 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.018196 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.018322 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.019421 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.020979 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.021778 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.021928 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.022056 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.022256 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.022338 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":108,"threads_started":1}
I20260812 11:16:44.022930 15104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.016510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.0.0.1:58372)
0812 11:16:44.017189 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.017203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.017295 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.017904 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.017907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.017949 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.018099 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.018107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.019547 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.019550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.020815 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.020828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.022135 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.022139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.022635 (+ 496us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.022660 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.022731 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.022804 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":439,"spinlock_wait_cycles":29824,"thread_start_us":92,"threads_started":1}
I20260812 11:16:44.023201 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.016684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.30.255.131:33589)
0812 11:16:44.017106 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.017118 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.017459 (+ 341us) server_negotiation.cc:407] Connection header received
0812 11:16:44.017541 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.017547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.017792 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.017928 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.018240 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.019416 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.020964 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.021990 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.022130 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.022255 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.022780 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.022872 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":118,"threads_started":1}
I20260812 11:16:44.024358 14380 tablet_service.cc:1467] Processing CreateTablet for tablet b5b951b26fbf4ec7a192be080895d047 (DEFAULT_TABLE table=test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.025264 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b951b26fbf4ec7a192be080895d047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.025595 14909 tablet_service.cc:1467] Processing CreateTablet for tablet b5b951b26fbf4ec7a192be080895d047 (DEFAULT_TABLE table=test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.025596 14648 tablet_service.cc:1467] Processing CreateTablet for tablet b5b951b26fbf4ec7a192be080895d047 (DEFAULT_TABLE table=test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.026609 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b951b26fbf4ec7a192be080895d047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.026616 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b951b26fbf4ec7a192be080895d047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.032521 15114 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Bootstrap starting.
I20260812 11:16:44.033890 15115 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Bootstrap starting.
I20260812 11:16:44.034689 15114 tablet_bootstrap.cc:654] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.034749 15116 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Bootstrap starting.
I20260812 11:16:44.035642 15114 log.cc:824] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:44.036010 15115 tablet_bootstrap.cc:654] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.036484 15116 tablet_bootstrap.cc:654] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.036739 15115 log.cc:824] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:44.037240 15116 log.cc:824] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:44.037756 15114 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: No bootstrap required, opened a new log
I20260812 11:16:44.037945 15114 ts_tablet_manager.cc:1397] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:16:44.038763 15115 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: No bootstrap required, opened a new log
I20260812 11:16:44.038939 15116 tablet_bootstrap.cc:492] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: No bootstrap required, opened a new log
I20260812 11:16:44.038949 15115 ts_tablet_manager.cc:1397] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 11:16:44.039115 15116 ts_tablet_manager.cc:1397] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 11:16:44.041911 15114 raft_consensus.cc:348] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.042181 15114 raft_consensus.cc:374] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.042061 15116 raft_consensus.cc:348] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.042268 15116 raft_consensus.cc:374] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.042268 15114 raft_consensus.cc:729] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.042326 15116 raft_consensus.cc:729] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.042716 15114 consensus_queue.cc:260] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.042759 15116 consensus_queue.cc:260] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.042663 15115 raft_consensus.cc:348] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.042874 15115 raft_consensus.cc:374] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.042948 15115 raft_consensus.cc:729] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.043412 15061 heartbeater.cc:503] Master 127.30.255.190:43543 was elected leader, sending a full tablet report...
I20260812 11:16:44.043390 15115 consensus_queue.cc:260] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.043697 15116 ts_tablet_manager.cc:1428] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:16:44.043725 15114 ts_tablet_manager.cc:1428] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:16:44.044056 14503 heartbeater.cc:503] Master 127.30.255.190:43543 was elected leader, sending a full tablet report...
I20260812 11:16:44.044178 15115 ts_tablet_manager.cc:1428] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 11:16:44.044436 14772 heartbeater.cc:503] Master 127.30.255.190:43543 was elected leader, sending a full tablet report...
I20260812 11:16:44.076638 15126 raft_consensus.cc:482] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:44.076808 15126 raft_consensus.cc:504] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.077826 15126 leader_election.cc:288] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.082854 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.078152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:59509 (local address 127.30.255.130:33217)
0812 11:16:44.078264 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.078268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.078734 (+ 466us) server_negotiation.cc:407] Connection header received
0812 11:16:44.078835 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.078837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.078880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.078955 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.079150 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.079852 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.081560 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.082195 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.082312 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.082363 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.082665 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.082730 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:16:44.082854 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.078083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45611 (local address 127.30.255.129:41011)
0812 11:16:44.078192 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.078196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.078211 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:16:44.078336 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.078338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.078381 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.078444 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.078755 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.079539 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.081568 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.082195 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.082312 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.082363 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.082665 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.082730 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:16:44.082882 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.078033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:33217 (local address 127.30.255.131:59509)
0812 11:16:44.078645 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.078658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.078751 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.078960 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.078962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.078974 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.079069 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.079072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.079971 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.079975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.081441 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.081453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.082313 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.082317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.082511 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.082573 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.082645 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.082722 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":432,"thread_start_us":89,"threads_started":1}
I20260812 11:16:44.082887 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.078020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41011 (local address 127.30.255.131:45611)
0812 11:16:44.078139 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.078155 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.078238 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.078488 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.078491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.078511 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.078646 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.078655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.079656 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.079659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.081441 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.081453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.082313 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.082317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.082529 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.082567 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.082651 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.082722 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":54}
I20260812 11:16:44.083446 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b951b26fbf4ec7a192be080895d047" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638" is_pre_election: true
I20260812 11:16:44.083648 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b951b26fbf4ec7a192be080895d047" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" is_pre_election: true
I20260812 11:16:44.083693 14687 raft_consensus.cc:2454] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.083890 14416 raft_consensus.cc:2454] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.084136 14828 leader_election.cc:302] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.084370 15126 raft_consensus.cc:2781] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:44.084455 15126 raft_consensus.cc:482] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:44.084504 15126 raft_consensus.cc:3037] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.085451 15126 raft_consensus.cc:504] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.085734 15126 leader_election.cc:288] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Requested vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.086056 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b951b26fbf4ec7a192be080895d047" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
I20260812 11:16:44.086074 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b951b26fbf4ec7a192be080895d047" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638"
I20260812 11:16:44.086175 14687 raft_consensus.cc:3037] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.086175 14416 raft_consensus.cc:3037] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.087347 14687 raft_consensus.cc:2454] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.087347 14416 raft_consensus.cc:2454] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.087644 14827 leader_election.cc:302] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 836b8cac1e3e4ab391feef2b7cbacb8f, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.087807 15126 raft_consensus.cc:2781] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:44.088030 15126 raft_consensus.cc:686] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 1 LEADER]: Becoming Leader. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Running, Role: LEADER
I20260812 11:16:44.088310 15126 consensus_queue.cc:237] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.091032 14119 catalog_manager.cc:5495] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b reported cstate change: term changed from 0 to 1, leader changed from <none> to e7149eddd07948e2b0303476c6c59e1b (127.30.255.131). New cstate: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:44.117889 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.114792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.30.255.131:33589)
0812 11:16:44.114897 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.114901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.114913 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:44.115038 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.115040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.115089 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.115177 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:44.115410 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.115874 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.116660 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.116894 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.116968 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.117175 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.117573 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.117671 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.117723 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.117777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:16:44.122912 14416 raft_consensus.cc:1261] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.123215 14687 raft_consensus.cc:1261] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.123570 15136 consensus_queue.cc:1035] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.123772 15126 consensus_queue.cc:1035] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.131219 15139 mvcc.cc:205] Tried to move back new op lower bound from 7317640823281442816 to 7317640823148105728. Current Snapshot: MvccSnapshot[applied={T|T < 7317640823281442816}]
I20260812 11:16:44.132527 15141 mvcc.cc:205] Tried to move back new op lower bound from 7317640823281442816 to 7317640823148105728. Current Snapshot: MvccSnapshot[applied={T|T < 7317640823281442816}]
I20260812 11:16:44.132730 15140 mvcc.cc:205] Tried to move back new op lower bound from 7317640823281442816 to 7317640823148105728. Current Snapshot: MvccSnapshot[applied={T|T < 7317640823281442816}]
I20260812 11:16:44.138785 14119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.138916 14119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.141192 14119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Sending DeleteTablet for 3 replicas of tablet b5b951b26fbf4ec7a192be080895d047
I20260812 11:16:44.141930 14648 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b951b26fbf4ec7a192be080895d047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47704
I20260812 11:16:44.142123 14380 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b951b26fbf4ec7a192be080895d047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260812 11:16:44.142536 15154 tablet_replica.cc:330] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: stopping tablet replica
I20260812 11:16:44.142448 14119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:16:44.142575 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b951b26fbf4ec7a192be080895d047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58372
W20260812 11:16:44.142835 14119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:44.142989 15154 raft_consensus.cc:2229] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.143085 15156 tablet_replica.cc:330] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: stopping tablet replica
I20260812 11:16:44.143388 15154 raft_consensus.cc:2258] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.143822 15155 tablet_replica.cc:330] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: stopping tablet replica
I20260812 11:16:44.144068 15155 raft_consensus.cc:2229] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.144399 15155 raft_consensus.cc:2258] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.145048 15154 ts_tablet_manager.cc:1905] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.143457 15156 raft_consensus.cc:2229] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:44.146770 15156 raft_consensus.cc:2258] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.147837 15154 ts_tablet_manager.cc:1918] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.147987 15154 log.cc:1196] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/b5b951b26fbf4ec7a192be080895d047
I20260812 11:16:44.148370 15154 ts_tablet_manager.cc:1939] T b5b951b26fbf4ec7a192be080895d047 P 687bc46e13ff4fac808cd77d9cb59638: Deleting consensus metadata
I20260812 11:16:44.148598 14648 tablet_service.cc:1467] Processing CreateTablet for tablet 759be5d5291046d6bed62f26b66cf29c (DEFAULT_TABLE table=test-table [id=a0c5c58575bf4e128c46112b95d296e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.148635 14380 tablet_service.cc:1467] Processing CreateTablet for tablet 759be5d5291046d6bed62f26b66cf29c (DEFAULT_TABLE table=test-table [id=a0c5c58575bf4e128c46112b95d296e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.148833 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759be5d5291046d6bed62f26b66cf29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.148883 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759be5d5291046d6bed62f26b66cf29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.150578 15115 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Bootstrap starting.
I20260812 11:16:44.150915 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 759be5d5291046d6bed62f26b66cf29c (DEFAULT_TABLE table=test-table [id=a0c5c58575bf4e128c46112b95d296e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.151170 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759be5d5291046d6bed62f26b66cf29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.151489 15115 tablet_bootstrap.cc:654] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.151669 14099 catalog_manager.cc:4841] TS 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217): tablet b5b951b26fbf4ec7a192be080895d047 (table test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]) successfully deleted
I20260812 11:16:44.152084 15114 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Bootstrap starting.
I20260812 11:16:44.152653 15156 ts_tablet_manager.cc:1905] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.152974 15114 tablet_bootstrap.cc:654] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.153513 15116 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Bootstrap starting.
I20260812 11:16:44.154363 15155 ts_tablet_manager.cc:1905] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.154410 15115 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: No bootstrap required, opened a new log
I20260812 11:16:44.154422 15116 tablet_bootstrap.cc:654] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.154480 15115 ts_tablet_manager.cc:1397] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 11:16:44.154939 15115 raft_consensus.cc:348] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.155052 15115 raft_consensus.cc:374] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.155087 15115 raft_consensus.cc:729] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.155201 15115 consensus_queue.cc:260] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.155476 15115 ts_tablet_manager.cc:1428] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.156526 15114 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: No bootstrap required, opened a new log
I20260812 11:16:44.156621 15114 ts_tablet_manager.cc:1397] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 11:16:44.157315 15114 raft_consensus.cc:348] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.157430 15156 ts_tablet_manager.cc:1918] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.157430 15114 raft_consensus.cc:374] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.157498 15114 raft_consensus.cc:729] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.157529 15156 log.cc:1196] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/b5b951b26fbf4ec7a192be080895d047
I20260812 11:16:44.157629 15114 consensus_queue.cc:260] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.157869 15114 ts_tablet_manager.cc:1428] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 11:16:44.158409 15155 ts_tablet_manager.cc:1918] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.158494 15155 log.cc:1196] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/b5b951b26fbf4ec7a192be080895d047
I20260812 11:16:44.158555 15156 ts_tablet_manager.cc:1939] T b5b951b26fbf4ec7a192be080895d047 P e7149eddd07948e2b0303476c6c59e1b: Deleting consensus metadata
I20260812 11:16:44.158787 15155 ts_tablet_manager.cc:1939] T b5b951b26fbf4ec7a192be080895d047 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting consensus metadata
I20260812 11:16:44.159682 14098 catalog_manager.cc:4841] TS e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589): tablet b5b951b26fbf4ec7a192be080895d047 (table test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]) successfully deleted
I20260812 11:16:44.159910 14098 catalog_manager.cc:4841] TS 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011): tablet b5b951b26fbf4ec7a192be080895d047 (table test-table [id=fedb2fcb4ff94297aa418ae1e92f12f5]) successfully deleted
I20260812 11:16:44.160367 15116 tablet_bootstrap.cc:492] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: No bootstrap required, opened a new log
I20260812 11:16:44.160454 15116 ts_tablet_manager.cc:1397] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 11:16:44.160908 15116 raft_consensus.cc:348] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.161011 15116 raft_consensus.cc:374] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.161039 15116 raft_consensus.cc:729] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.161146 15116 consensus_queue.cc:260] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.161381 15116 ts_tablet_manager.cc:1428] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260812 11:16:44.208715 15062 tablet.cc:2369] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:44.249110 14773 tablet.cc:2369] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:16:44.289915 14505 tablet.cc:2369] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:16:44.364333 15136 raft_consensus.cc:482] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:44.364482 15136 raft_consensus.cc:504] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.364863 15136 leader_election.cc:288] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.365293 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759be5d5291046d6bed62f26b66cf29c" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" is_pre_election: true
I20260812 11:16:44.365298 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759be5d5291046d6bed62f26b66cf29c" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638" is_pre_election: true
I20260812 11:16:44.365444 14687 raft_consensus.cc:2454] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.365435 14416 raft_consensus.cc:2454] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.365747 14827 leader_election.cc:302] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 836b8cac1e3e4ab391feef2b7cbacb8f, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.365919 15136 raft_consensus.cc:2781] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:44.365989 15136 raft_consensus.cc:482] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:44.366027 15136 raft_consensus.cc:3037] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.367012 15136 raft_consensus.cc:504] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.367349 15136 leader_election.cc:288] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Requested vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.367679 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759be5d5291046d6bed62f26b66cf29c" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
I20260812 11:16:44.367727 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759be5d5291046d6bed62f26b66cf29c" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638"
I20260812 11:16:44.367784 14416 raft_consensus.cc:3037] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.367830 14687 raft_consensus.cc:3037] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.368686 14416 raft_consensus.cc:2454] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.368768 14687 raft_consensus.cc:2454] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.368945 14827 leader_election.cc:302] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 836b8cac1e3e4ab391feef2b7cbacb8f, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.369158 15136 raft_consensus.cc:2781] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:44.369232 15136 raft_consensus.cc:686] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 1 LEADER]: Becoming Leader. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Running, Role: LEADER
I20260812 11:16:44.369359 15136 consensus_queue.cc:237] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.370913 14119 catalog_manager.cc:5495] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b reported cstate change: term changed from 0 to 1, leader changed from <none> to e7149eddd07948e2b0303476c6c59e1b (127.30.255.131). New cstate: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } health_report { overall_health: HEALTHY } } }
Aug 12 11:16:44 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533403, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.190@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:44.408138 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.400828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.30.255.190:43543)
0812 11:16:44.400951 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.400956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.401108 (+ 152us) server_negotiation.cc:407] Connection header received
0812 11:16:44.401206 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.401208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.401246 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.401300 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:44.401886 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.402375 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.403093 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.403306 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.403370 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.405431 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:44.405435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:44.405437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:44.405458 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:44.406740 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.407148 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.407150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.407151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.407217 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.407446 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.407448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.407450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.407620 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:44.407694 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.407923 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.408018 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":50}
I20260812 11:16:44.408244 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.400754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43543 (local address 127.0.0.1:42812)
0812 11:16:44.401029 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.401042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.401134 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.401325 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.401327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.401692 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:44.401791 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.401797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.402452 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.402454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.403012 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.403019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.403378 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.403380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.403469 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.403949 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:44.403961 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:44.405309 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:44.406832 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:44.406841 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:44.406845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:44.407076 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:44.407301 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:44.407304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:44.407305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:44.407380 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:44.407696 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:44.407701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:44.407800 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.407884 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.408087 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":76,"thread_start_us":85,"threads_started":1}
I20260812 11:16:44.411576 14119 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42812:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:16:44.412227 14119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:44.416554 14380 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf57114196472f98a9f0c9e89c754f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ec3424f00ff4ea1b9d725e107e7b76c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:44.416568 14648 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf57114196472f98a9f0c9e89c754f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ec3424f00ff4ea1b9d725e107e7b76c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:44.416935 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf57114196472f98a9f0c9e89c754f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.416935 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf57114196472f98a9f0c9e89c754f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.416851 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf57114196472f98a9f0c9e89c754f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ec3424f00ff4ea1b9d725e107e7b76c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:16:44.417172 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf57114196472f98a9f0c9e89c754f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.419392 15116 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: Bootstrap starting.
I20260812 11:16:44.419788 15115 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: Bootstrap starting.
I20260812 11:16:44.420239 15116 tablet_bootstrap.cc:654] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.420293 15114 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: Bootstrap starting.
I20260812 11:16:44.420679 15115 tablet_bootstrap.cc:654] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.421195 15114 tablet_bootstrap.cc:654] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.421409 15116 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: No bootstrap required, opened a new log
I20260812 11:16:44.421499 15116 ts_tablet_manager.cc:1397] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:16:44.421797 15115 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: No bootstrap required, opened a new log
I20260812 11:16:44.421893 15115 ts_tablet_manager.cc:1397] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:44.421921 15116 raft_consensus.cc:348] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.422020 15116 raft_consensus.cc:374] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.422046 15116 raft_consensus.cc:729] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.422170 15116 consensus_queue.cc:260] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.422299 15115 raft_consensus.cc:348] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.422389 15115 raft_consensus.cc:374] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.422418 15115 raft_consensus.cc:729] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.422428 15116 ts_tablet_manager.cc:1428] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.422549 15114 tablet_bootstrap.cc:492] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: No bootstrap required, opened a new log
I20260812 11:16:44.422528 15115 consensus_queue.cc:260] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.422628 15114 ts_tablet_manager.cc:1397] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:44.422633 15136 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.422755 15136 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.422806 15115 ts_tablet_manager.cc:1428] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.422989 15114 raft_consensus.cc:348] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.423098 15114 raft_consensus.cc:374] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.423147 15114 raft_consensus.cc:729] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.423280 15114 consensus_queue.cc:260] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.423301 15130 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.423439 15130 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.423543 15114 ts_tablet_manager.cc:1428] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.423523 15128 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.423650 15128 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.522076 15136 raft_consensus.cc:482] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:44.522193 15136 raft_consensus.cc:504] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.522563 15136 leader_election.cc:288] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.522948 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf57114196472f98a9f0c9e89c754f" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" is_pre_election: true
I20260812 11:16:44.522959 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf57114196472f98a9f0c9e89c754f" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638" is_pre_election: true
I20260812 11:16:44.523088 14416 raft_consensus.cc:2454] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.523094 14687 raft_consensus.cc:2454] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 0.
I20260812 11:16:44.523371 14828 leader_election.cc:302] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.523577 15136 raft_consensus.cc:2781] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:44.523635 15136 raft_consensus.cc:482] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:44.523674 15136 raft_consensus.cc:3037] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.524600 15136 raft_consensus.cc:504] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.524924 15136 leader_election.cc:288] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Requested vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217)
I20260812 11:16:44.525238 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf57114196472f98a9f0c9e89c754f" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
I20260812 11:16:44.525250 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf57114196472f98a9f0c9e89c754f" candidate_uuid: "e7149eddd07948e2b0303476c6c59e1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638"
I20260812 11:16:44.525343 14687 raft_consensus.cc:3037] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.525348 14416 raft_consensus.cc:3037] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.526170 14687 raft_consensus.cc:2454] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.526186 14416 raft_consensus.cc:2454] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7149eddd07948e2b0303476c6c59e1b in term 1.
I20260812 11:16:44.526441 14828 leader_election.cc:302] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, e7149eddd07948e2b0303476c6c59e1b; no voters:
I20260812 11:16:44.526592 15136 raft_consensus.cc:2781] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:44.526684 15136 raft_consensus.cc:686] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [term 1 LEADER]: Becoming Leader. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Running, Role: LEADER
I20260812 11:16:44.526823 15136 consensus_queue.cc:237] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.527431 15126 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: TxnStatusTablet state changed. Reason: New leader e7149eddd07948e2b0303476c6c59e1b. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.527522 15126 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:44.527730 15177 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:16:44.527896 15177 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:16:44.528403 14121 catalog_manager.cc:5495] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b reported cstate change: term changed from 0 to 1, leader changed from <none> to e7149eddd07948e2b0303476c6c59e1b (127.30.255.131). New cstate: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } health_report { overall_health: HEALTHY } } }
I20260812 11:16:44.546653 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.542839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.30.255.131:33589)
0812 11:16:44.542939 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.542943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.543012 (+ 69us) server_negotiation.cc:407] Connection header received
0812 11:16:44.543159 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.543162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.543214 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.543326 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:44.543568 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.544336 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.545517 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.545867 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.545966 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.546240 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.546322 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.546389 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.546468 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.546520 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:16:44.546666 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.542781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.0.0.1:58396)
0812 11:16:44.542923 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.542939 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.543052 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.543347 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.543350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.543357 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:44.543466 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.543472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.544444 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.544448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.545402 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.545414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.545971 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.545974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.546069 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.546121 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.546394 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.546403 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.546451 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.546528 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 11:16:44.549410 14416 raft_consensus.cc:1261] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.549669 14687 raft_consensus.cc:1261] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.550163 15136 consensus_queue.cc:1035] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.550767 15136 consensus_queue.cc:1035] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.551431 15128 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: TxnStatusTablet state changed. Reason: New leader e7149eddd07948e2b0303476c6c59e1b. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.551530 15128 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.552246 15130 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: TxnStatusTablet state changed. Reason: New leader e7149eddd07948e2b0303476c6c59e1b. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.552356 15130 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.554605 15136 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.554708 15136 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:44.555445 15136 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.555538 15136 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P e7149eddd07948e2b0303476c6c59e1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:16:44.557129 15130 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.557224 15130 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 687bc46e13ff4fac808cd77d9cb59638: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.557272 15128 tablet_replica.cc:439] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7149eddd07948e2b0303476c6c59e1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } } }
I20260812 11:16:44.557369 15128 tablet_replica.cc:442] T 16bf57114196472f98a9f0c9e89c754f P 836b8cac1e3e4ab391feef2b7cbacb8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:16:44.568063 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.564754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.30.255.131:60499)
0812 11:16:44.565063 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.565075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.565160 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.565349 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.565351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.565356 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:44.565434 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.565437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.566310 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.566312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.566911 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.566918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.567441 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.567443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.567540 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.567580 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.567833 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.567837 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.567876 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.567936 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260812 11:16:44.568068 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.564825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:60499 (local address 127.30.255.131:33589)
0812 11:16:44.564926 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.564931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.565135 (+ 204us) server_negotiation.cc:407] Connection header received
0812 11:16:44.565223 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.565226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.565271 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.565345 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:44.565496 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.566230 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.567002 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.567349 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.567442 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.567694 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.567763 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.567828 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.567887 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.567932 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":35}
I20260812 11:16:44.573351 14416 raft_consensus.cc:1261] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.573470 14687 raft_consensus.cc:1261] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Refusing update from remote peer e7149eddd07948e2b0303476c6c59e1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.573688 15187 consensus_queue.cc:1035] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.573833 15126 consensus_queue.cc:1035] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.606779 14118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.606899 14118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.608713 14118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Sending DeleteTablet for 3 replicas of tablet 759be5d5291046d6bed62f26b66cf29c
I20260812 11:16:44.609254 14380 tablet_service.cc:1514] Processing DeleteTablet for tablet 759be5d5291046d6bed62f26b66cf29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260812 11:16:44.609441 14648 tablet_service.cc:1514] Processing DeleteTablet for tablet 759be5d5291046d6bed62f26b66cf29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47704
I20260812 11:16:44.609606 15154 tablet_replica.cc:330] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: stopping tablet replica
I20260812 11:16:44.609619 15155 tablet_replica.cc:330] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: stopping tablet replica
I20260812 11:16:44.609720 15154 raft_consensus.cc:2229] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.609730 15155 raft_consensus.cc:2229] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.609809 15154 raft_consensus.cc:2258] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.609828 15155 raft_consensus.cc:2258] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.610390 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 759be5d5291046d6bed62f26b66cf29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 11:16:44.610494 15154 ts_tablet_manager.cc:1905] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.610543 15156 tablet_replica.cc:330] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: stopping tablet replica
I20260812 11:16:44.610649 15156 raft_consensus.cc:2229] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:44.610651 15155 ts_tablet_manager.cc:1905] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.610809 15156 raft_consensus.cc:2258] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.611543 15156 ts_tablet_manager.cc:1905] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.613353 15155 ts_tablet_manager.cc:1918] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:44.613421 15155 log.cc:1196] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/759be5d5291046d6bed62f26b66cf29c
I20260812 11:16:44.613660 15155 ts_tablet_manager.cc:1939] T 759be5d5291046d6bed62f26b66cf29c P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting consensus metadata
I20260812 11:16:44.614204 15156 ts_tablet_manager.cc:1918] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:44.614276 15156 log.cc:1196] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/759be5d5291046d6bed62f26b66cf29c
I20260812 11:16:44.614285 14098 catalog_manager.cc:4841] TS 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011): tablet 759be5d5291046d6bed62f26b66cf29c (table test-table [id=a0c5c58575bf4e128c46112b95d296e6]) successfully deleted
I20260812 11:16:44.614518 15156 ts_tablet_manager.cc:1939] T 759be5d5291046d6bed62f26b66cf29c P e7149eddd07948e2b0303476c6c59e1b: Deleting consensus metadata
I20260812 11:16:44.615077 14098 catalog_manager.cc:4841] TS e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589): tablet 759be5d5291046d6bed62f26b66cf29c (table test-table [id=a0c5c58575bf4e128c46112b95d296e6]) successfully deleted
I20260812 11:16:44.616959 15154 ts_tablet_manager.cc:1918] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:44.617036 15154 log.cc:1196] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/759be5d5291046d6bed62f26b66cf29c
I20260812 11:16:44.617260 15154 ts_tablet_manager.cc:1939] T 759be5d5291046d6bed62f26b66cf29c P 687bc46e13ff4fac808cd77d9cb59638: Deleting consensus metadata
I20260812 11:16:44.617918 14099 catalog_manager.cc:4841] TS 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217): tablet 759be5d5291046d6bed62f26b66cf29c (table test-table [id=a0c5c58575bf4e128c46112b95d296e6]) successfully deleted
Aug 12 11:16:44 dist-test-slave-gjrz krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:16:44 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:44 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:16:44.638144 14219 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:44.631969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.30.255.190:43543)
0812 11:16:44.632095 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.632100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.632262 (+ 162us) server_negotiation.cc:407] Connection header received
0812 11:16:44.632371 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.632374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.632420 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.632504 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:44.633104 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.633902 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.634744 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.635079 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.635181 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.638023 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42824: BlockingRecv error: recv got EOF from 127.0.0.1:42824 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:16:44 dist-test-slave-gjrz krb5kdc[14042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM
I20260812 11:16:44.647486 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.639637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.30.255.190:43543)
0812 11:16:44.639738 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.639742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.639920 (+ 178us) server_negotiation.cc:407] Connection header received
0812 11:16:44.640024 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.640027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.640068 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.640139 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:44.640700 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.641234 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.641997 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.642239 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.642322 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.644678 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:44.644681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:44.644683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:44.644706 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:44.645935 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.646392 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.646394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.646396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.646462 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:44.646739 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:44.646741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:44.646742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:44.646891 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:44.646981 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.647228 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.647368 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:16:44.650230 14118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:44.650784 14118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:44.655673 14380 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 (DEFAULT_TABLE table=test-table [id=e948baa327e64a628adca87cdf1f6b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.655833 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 (DEFAULT_TABLE table=test-table [id=e948baa327e64a628adca87cdf1f6b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.655959 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc9d5045314b1ba5c56102c00f90f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.656103 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc9d5045314b1ba5c56102c00f90f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.657788 15114 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Bootstrap starting.
I20260812 11:16:44.658582 15116 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Bootstrap starting.
I20260812 11:16:44.658655 15114 tablet_bootstrap.cc:654] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.658697 14648 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 (DEFAULT_TABLE table=test-table [id=e948baa327e64a628adca87cdf1f6b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.658963 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc9d5045314b1ba5c56102c00f90f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.659371 15116 tablet_bootstrap.cc:654] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.660470 15116 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: No bootstrap required, opened a new log
I20260812 11:16:44.660537 15116 ts_tablet_manager.cc:1397] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:44.660885 15116 raft_consensus.cc:348] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.660987 15116 raft_consensus.cc:374] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.661012 15116 raft_consensus.cc:729] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.661114 15116 consensus_queue.cc:260] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.661334 15116 ts_tablet_manager.cc:1428] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:44.661531 15115 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Bootstrap starting.
I20260812 11:16:44.662006 15114 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: No bootstrap required, opened a new log
I20260812 11:16:44.662088 15114 ts_tablet_manager.cc:1397] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 11:16:44.662379 15115 tablet_bootstrap.cc:654] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.662503 15114 raft_consensus.cc:348] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.662611 15114 raft_consensus.cc:374] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.662657 15114 raft_consensus.cc:729] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.662781 15114 consensus_queue.cc:260] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.663034 15114 ts_tablet_manager.cc:1428] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.663657 15115 tablet_bootstrap.cc:492] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: No bootstrap required, opened a new log
I20260812 11:16:44.663743 15115 ts_tablet_manager.cc:1397] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:44.664193 15115 raft_consensus.cc:348] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.664304 15115 raft_consensus.cc:374] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.664337 15115 raft_consensus.cc:729] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.664450 15115 consensus_queue.cc:260] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.664723 15115 ts_tablet_manager.cc:1428] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.733356 15128 raft_consensus.cc:482] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:44.733502 15128 raft_consensus.cc:504] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.734572 15128 leader_election.cc:288] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217), e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:44.736634 15130 raft_consensus.cc:482] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:44.736752 15130 raft_consensus.cc:504] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.737898 15130 leader_election.cc:288] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:44.739705 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.734806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:33217 (local address 127.30.255.129:51775)
0812 11:16:44.735152 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.735167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.735293 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.735547 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.735550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.735565 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.735733 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.735742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.736664 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.736668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.737866 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.737879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.739198 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.739205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.739328 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.739364 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.739440 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.739523 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":21,"thread_start_us":64,"threads_started":1}
I20260812 11:16:44.740175 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.734872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:51775 (local address 127.30.255.130:33217)
0812 11:16:44.735205 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.735210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.735224 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:44.735407 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.735410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.735459 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.735529 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.735872 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.736538 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.738011 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.738597 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.738750 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.738810 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.739942 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.740030 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":80,"threads_started":1}
I20260812 11:16:44.740599 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638" is_pre_election: true
I20260812 11:16:44.740742 14687 raft_consensus.cc:2454] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 836b8cac1e3e4ab391feef2b7cbacb8f in term 0.
I20260812 11:16:44.741266 14296 leader_election.cc:302] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, 836b8cac1e3e4ab391feef2b7cbacb8f; no voters:
I20260812 11:16:44.741582 15128 raft_consensus.cc:2781] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:44.741637 15128 raft_consensus.cc:482] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:44.741672 15128 raft_consensus.cc:3037] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.741739 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.734913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:39685 (local address 127.30.255.131:33589)
0812 11:16:44.735033 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.735038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.735281 (+ 243us) server_negotiation.cc:407] Connection header received
0812 11:16:44.735456 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.735460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.735512 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.735595 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.735876 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.736578 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.740461 (+ 3883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.741049 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.741131 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.741183 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.741526 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.741611 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:44.742136 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.734811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.30.255.129:39685)
0812 11:16:44.735173 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.735187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.735310 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.735605 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.735608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.735624 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.735775 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.735781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.739175 (+ 3394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.739178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.740326 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.740338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.741278 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.741281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.741378 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.741404 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.741921 (+ 517us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.741997 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":98,"thread_start_us":92,"threads_started":1}
I20260812 11:16:44.742596 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7149eddd07948e2b0303476c6c59e1b" is_pre_election: true
I20260812 11:16:44.742861 14972 raft_consensus.cc:2454] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 836b8cac1e3e4ab391feef2b7cbacb8f in term 0.
I20260812 11:16:44.743000 15128 raft_consensus.cc:504] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.743368 15128 leader_election.cc:288] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [CANDIDATE]: Term 1 election: Requested vote from peers 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217), e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:44.743734 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7149eddd07948e2b0303476c6c59e1b"
I20260812 11:16:44.743794 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.738550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.30.255.130:45403)
0812 11:16:44.738821 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.738834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.738930 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.739564 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.739567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.739588 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.739698 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.739704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.740956 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.740960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.742193 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.742204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.743366 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.743370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.743475 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.743500 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.743577 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.743648 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260812 11:16:44.743839 14972 raft_consensus.cc:3037] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.743842 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.738206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41011 (local address 127.30.255.130:52853)
0812 11:16:44.738452 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:44.738466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:44.738562 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:44.739681 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:44.739683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:44.739704 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:16:44.740386 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.740396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.741342 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.741345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.742478 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:44.742490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.743429 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.743433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:44.743540 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.743563 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:16:44.743636 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 11:16:44.743706 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260812 11:16:44.744171 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.738957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:52853 (local address 127.30.255.129:41011)
0812 11:16:44.739413 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.739422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.739434 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:44.739470 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.739473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.739511 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.739582 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.740515 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.741196 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.742616 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.743183 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.743317 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.743362 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.743969 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.744053 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":124,"threads_started":1}
I20260812 11:16:44.744455 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "687bc46e13ff4fac808cd77d9cb59638"
I20260812 11:16:44.744524 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.738970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:45403 (local address 127.30.255.131:33589)
0812 11:16:44.739296 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.739301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.739313 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:44.739391 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.739395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.739441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.739883 (+ 442us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:16:44.739979 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.740839 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.742467 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.743191 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.744128 (+ 937us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.744175 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.744208 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.744416 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":91,"threads_started":1}
I20260812 11:16:44.744568 14687 raft_consensus.cc:3037] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:44.745081 14972 raft_consensus.cc:2454] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 836b8cac1e3e4ab391feef2b7cbacb8f in term 1.
I20260812 11:16:44.745592 14687 raft_consensus.cc:2454] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 836b8cac1e3e4ab391feef2b7cbacb8f in term 1.
I20260812 11:16:44.745884 14296 leader_election.cc:302] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, 836b8cac1e3e4ab391feef2b7cbacb8f; no voters:
I20260812 11:16:44.745956 14415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" is_pre_election: true
I20260812 11:16:44.746054 15128 raft_consensus.cc:2781] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:44.746230 15128 raft_consensus.cc:686] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 LEADER]: Becoming Leader. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Running, Role: LEADER
I20260812 11:16:44.746479 15128 consensus_queue.cc:237] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.746587 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc9d5045314b1ba5c56102c00f90f8" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7149eddd07948e2b0303476c6c59e1b" is_pre_election: true
I20260812 11:16:44.746775 14972 raft_consensus.cc:2379] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 687bc46e13ff4fac808cd77d9cb59638 in current term 1: Already voted for candidate 836b8cac1e3e4ab391feef2b7cbacb8f in this term.
I20260812 11:16:44.756896 14547 leader_election.cc:302] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638; no voters: 836b8cac1e3e4ab391feef2b7cbacb8f, e7149eddd07948e2b0303476c6c59e1b
I20260812 11:16:44.756927 14121 catalog_manager.cc:5495] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129). New cstate: current_term: 1 leader_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:44.757367 15130 raft_consensus.cc:2726] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:16:44.771481 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:44.768002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.30.255.129:41011)
0812 11:16:44.768122 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:44.768126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:44.768139 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:44.768274 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:44.768277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:44.768330 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:44.768416 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:44.768656 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.769156 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.769986 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:44.770213 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:44.770298 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:44.770565 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:44.770976 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:44.771088 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:44.771160 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 11:16:44.771211 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:16:44.774309 14972 raft_consensus.cc:1261] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Refusing update from remote peer 836b8cac1e3e4ab391feef2b7cbacb8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.774744 14687 raft_consensus.cc:1261] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Refusing update from remote peer 836b8cac1e3e4ab391feef2b7cbacb8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:44.774838 15128 consensus_queue.cc:1035] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.775118 15246 consensus_queue.cc:1035] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:44.785983 14121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.786131 14121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:44.788007 14121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Sending DeleteTablet for 3 replicas of tablet 97cc9d5045314b1ba5c56102c00f90f8
I20260812 11:16:44.788506 14380 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260812 11:16:44.788556 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 11:16:44.788674 15155 tablet_replica.cc:330] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: stopping tablet replica
I20260812 11:16:44.788763 15155 raft_consensus.cc:2229] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:44.788962 15156 tablet_replica.cc:330] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: stopping tablet replica
I20260812 11:16:44.788982 15155 raft_consensus.cc:2258] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.789074 15156 raft_consensus.cc:2229] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.789202 15156 raft_consensus.cc:2258] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.789600 14648 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cc9d5045314b1ba5c56102c00f90f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:44 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47704
I20260812 11:16:44.789690 15155 ts_tablet_manager.cc:1905] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.789770 15154 tablet_replica.cc:330] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: stopping tablet replica
I20260812 11:16:44.789876 15154 raft_consensus.cc:2229] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:44.789994 15154 raft_consensus.cc:2258] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:44.790475 15156 ts_tablet_manager.cc:1905] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.791939 15155 ts_tablet_manager.cc:1918] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.792018 15155 log.cc:1196] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/97cc9d5045314b1ba5c56102c00f90f8
I20260812 11:16:44.792233 15155 ts_tablet_manager.cc:1939] T 97cc9d5045314b1ba5c56102c00f90f8 P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting consensus metadata
I20260812 11:16:44.792784 14098 catalog_manager.cc:4841] TS 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011): tablet 97cc9d5045314b1ba5c56102c00f90f8 (table test-table [id=e948baa327e64a628adca87cdf1f6b58]) successfully deleted
I20260812 11:16:44.792943 15156 ts_tablet_manager.cc:1918] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.793026 15156 log.cc:1196] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/97cc9d5045314b1ba5c56102c00f90f8
I20260812 11:16:44.793304 15156 ts_tablet_manager.cc:1939] T 97cc9d5045314b1ba5c56102c00f90f8 P e7149eddd07948e2b0303476c6c59e1b: Deleting consensus metadata
I20260812 11:16:44.793946 14098 catalog_manager.cc:4841] TS e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589): tablet 97cc9d5045314b1ba5c56102c00f90f8 (table test-table [id=e948baa327e64a628adca87cdf1f6b58]) successfully deleted
I20260812 11:16:44.794389 15154 ts_tablet_manager.cc:1905] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:44.795420 14121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:16:44.795781 14121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:16:44.796722 15154 ts_tablet_manager.cc:1918] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:16:44.796803 15154 log.cc:1196] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/97cc9d5045314b1ba5c56102c00f90f8
I20260812 11:16:44.797052 15154 ts_tablet_manager.cc:1939] T 97cc9d5045314b1ba5c56102c00f90f8 P 687bc46e13ff4fac808cd77d9cb59638: Deleting consensus metadata
I20260812 11:16:44.797698 14099 catalog_manager.cc:4841] TS 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217): tablet 97cc9d5045314b1ba5c56102c00f90f8 (table test-table [id=e948baa327e64a628adca87cdf1f6b58]) successfully deleted
I20260812 11:16:44.799960 14380 tablet_service.cc:1467] Processing CreateTablet for tablet 929c78d081f84a66831be9803594fbcd (DEFAULT_TABLE table=test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.800235 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929c78d081f84a66831be9803594fbcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.800345 14648 tablet_service.cc:1467] Processing CreateTablet for tablet 929c78d081f84a66831be9803594fbcd (DEFAULT_TABLE table=test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.800566 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929c78d081f84a66831be9803594fbcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.801764 15114 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Bootstrap starting.
I20260812 11:16:44.802572 15115 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Bootstrap starting.
I20260812 11:16:44.802605 15114 tablet_bootstrap.cc:654] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.803236 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 929c78d081f84a66831be9803594fbcd (DEFAULT_TABLE table=test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:16:44.803354 15115 tablet_bootstrap.cc:654] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.803503 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929c78d081f84a66831be9803594fbcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:44.804611 15115 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: No bootstrap required, opened a new log
I20260812 11:16:44.804697 15115 ts_tablet_manager.cc:1397] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:44.805081 15115 raft_consensus.cc:348] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.805192 15115 raft_consensus.cc:374] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.805222 15115 raft_consensus.cc:729] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.805276 15116 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Bootstrap starting.
I20260812 11:16:44.805325 15115 consensus_queue.cc:260] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.805547 15115 ts_tablet_manager.cc:1428] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:16:44.805853 15114 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: No bootstrap required, opened a new log
I20260812 11:16:44.805925 15114 ts_tablet_manager.cc:1397] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 11:16:44.806071 15116 tablet_bootstrap.cc:654] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:44.806346 15114 raft_consensus.cc:348] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.806445 15114 raft_consensus.cc:374] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.806475 15114 raft_consensus.cc:729] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836b8cac1e3e4ab391feef2b7cbacb8f, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.806599 15114 consensus_queue.cc:260] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.806836 15114 ts_tablet_manager.cc:1428] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:16:44.807348 15116 tablet_bootstrap.cc:492] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: No bootstrap required, opened a new log
I20260812 11:16:44.807430 15116 ts_tablet_manager.cc:1397] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 11:16:44.807792 15116 raft_consensus.cc:348] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.807894 15116 raft_consensus.cc:374] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:44.807925 15116 raft_consensus.cc:729] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7149eddd07948e2b0303476c6c59e1b, State: Initialized, Role: FOLLOWER
I20260812 11:16:44.808048 15116 consensus_queue.cc:260] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:44.808302 15116 ts_tablet_manager.cc:1428] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:16:45.014397 15130 raft_consensus.cc:482] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:16:45.014526 15130 raft_consensus.cc:504] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:45.014886 15130 leader_election.cc:288] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:45.015380 14415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929c78d081f84a66831be9803594fbcd" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" is_pre_election: true
I20260812 11:16:45.015398 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929c78d081f84a66831be9803594fbcd" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7149eddd07948e2b0303476c6c59e1b" is_pre_election: true
I20260812 11:16:45.015528 14972 raft_consensus.cc:2454] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 687bc46e13ff4fac808cd77d9cb59638 in term 0.
I20260812 11:16:45.015532 14415 raft_consensus.cc:2454] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 687bc46e13ff4fac808cd77d9cb59638 in term 0.
I20260812 11:16:45.015832 14547 leader_election.cc:302] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, 836b8cac1e3e4ab391feef2b7cbacb8f; no voters:
I20260812 11:16:45.015993 15130 raft_consensus.cc:2781] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:16:45.016057 15130 raft_consensus.cc:482] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:16:45.016098 15130 raft_consensus.cc:3037] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:45.017112 15130 raft_consensus.cc:504] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:45.017414 15130 leader_election.cc:288] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 election: Requested vote from peers 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011), e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589)
I20260812 11:16:45.017743 14415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929c78d081f84a66831be9803594fbcd" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f"
I20260812 11:16:45.017850 14415 raft_consensus.cc:3037] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:45.017813 14972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929c78d081f84a66831be9803594fbcd" candidate_uuid: "687bc46e13ff4fac808cd77d9cb59638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7149eddd07948e2b0303476c6c59e1b"
I20260812 11:16:45.017928 14972 raft_consensus.cc:3037] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:45.018774 14415 raft_consensus.cc:2454] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 687bc46e13ff4fac808cd77d9cb59638 in term 1.
I20260812 11:16:45.018848 14972 raft_consensus.cc:2454] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 687bc46e13ff4fac808cd77d9cb59638 in term 1.
I20260812 11:16:45.019116 14547 leader_election.cc:302] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 687bc46e13ff4fac808cd77d9cb59638, 836b8cac1e3e4ab391feef2b7cbacb8f; no voters:
I20260812 11:16:45.019310 15130 raft_consensus.cc:2781] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:45.019511 15130 raft_consensus.cc:686] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 1 LEADER]: Becoming Leader. State: Replica: 687bc46e13ff4fac808cd77d9cb59638, State: Running, Role: LEADER
I20260812 11:16:45.019752 15130 consensus_queue.cc:237] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } }
I20260812 11:16:45.021978 14118 catalog_manager.cc:5495] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 reported cstate change: term changed from 0 to 1, leader changed from <none> to 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "687bc46e13ff4fac808cd77d9cb59638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "687bc46e13ff4fac808cd77d9cb59638" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 33217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 } health_report { overall_health: UNKNOWN } } }
I20260812 11:16:45.046849 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:45.043890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.30.255.130:33217)
0812 11:16:45.043995 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:45.044000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:45.044012 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:45.044160 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:45.044162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:45.044212 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:45.044299 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:45.044458 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.044964 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.045685 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.045884 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.045949 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:45.046137 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:45.046566 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:45.046655 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:45.046679 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 11:16:45.046726 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:16:45.056396 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:45.053296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:50751 (local address 127.30.255.131:33589)
0812 11:16:45.053419 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:45.053424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:45.053558 (+ 134us) server_negotiation.cc:407] Connection header received
0812 11:16:45.053692 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:45.053695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:45.053740 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:45.053796 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:45.054019 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.054512 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.055364 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.055624 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.055715 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:45.055979 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:45.056051 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:45.056106 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:45.056220 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 11:16:45.056273 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:16:45.056421 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:45.053190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33589 (local address 127.30.255.130:50751)
0812 11:16:45.053478 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:45.053496 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:45.053591 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:45.053813 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:45.053816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:45.053824 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:45.053926 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:45.053930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.054610 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.054613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:45.055236 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:45.055246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.055726 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.055729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:45.055816 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:45.055857 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:45.056130 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:45.056136 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:16:45.056190 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 11:16:45.056266 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260812 11:16:45.061821 14415 raft_consensus.cc:1261] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Refusing update from remote peer 687bc46e13ff4fac808cd77d9cb59638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:45.061810 14972 raft_consensus.cc:1261] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Refusing update from remote peer 687bc46e13ff4fac808cd77d9cb59638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:16:45.062232 15130 consensus_queue.cc:1035] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "836b8cac1e3e4ab391feef2b7cbacb8f" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:45.062638 15265 consensus_queue.cc:1035] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7149eddd07948e2b0303476c6c59e1b" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:16:45.079016 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:45.075909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:37155 (local address 127.30.255.130:33217)
0812 11:16:45.076012 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:45.076016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:45.076377 (+ 361us) server_negotiation.cc:407] Connection header received
0812 11:16:45.076470 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:45.076473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:45.076515 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:45.076576 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:16:45.076800 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.077330 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.078089 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.078302 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.078370 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:45.078625 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:16:45.078696 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:16:45.078759 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:45.078848 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 11:16:45.078902 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:16:45.079018 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:45.075846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:33217 (local address 127.30.255.131:37155)
0812 11:16:45.076292 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:45.076304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:45.076396 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:45.076593 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:45.076596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:45.076603 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:16:45.076703 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:45.076709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.077423 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.077426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:45.078004 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:45.078013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:45.078395 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:45.078398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:45.078483 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:45.078517 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:16:45.078775 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:16:45.078781 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:16:45.078834 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 11:16:45.078901 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":63,"threads_started":1}
I20260812 11:16:45.101914 14119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:45.102048 14119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:16:45.103982 14119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3002a2fb104942daaadc5a5d5105f6e5: Sending DeleteTablet for 3 replicas of tablet 929c78d081f84a66831be9803594fbcd
I20260812 11:16:45.104553 14648 tablet_service.cc:1514] Processing DeleteTablet for tablet 929c78d081f84a66831be9803594fbcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:45 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47704
I20260812 11:16:45.104552 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 929c78d081f84a66831be9803594fbcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:45 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 11:16:45.104678 14380 tablet_service.cc:1514] Processing DeleteTablet for tablet 929c78d081f84a66831be9803594fbcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:16:45 UTC) from {username='oryx', principal='oryx/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260812 11:16:45.104771 15156 tablet_replica.cc:330] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: stopping tablet replica
I20260812 11:16:45.104864 15155 tablet_replica.cc:330] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: stopping tablet replica
I20260812 11:16:45.104900 15156 raft_consensus.cc:2229] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:45.105022 15155 raft_consensus.cc:2229] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:16:45.105031 15156 raft_consensus.cc:2258] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:45.105141 15155 raft_consensus.cc:2258] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:45.105192 15154 tablet_replica.cc:330] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: stopping tablet replica
I20260812 11:16:45.105295 15154 raft_consensus.cc:2229] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:16:45.105542 15154 raft_consensus.cc:2258] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:16:45.105916 15155 ts_tablet_manager.cc:1905] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:45.105949 15156 ts_tablet_manager.cc:1905] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:45.106326 15154 ts_tablet_manager.cc:1905] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:16:45.108429 15156 ts_tablet_manager.cc:1918] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:45.108429 15155 ts_tablet_manager.cc:1918] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:45.108507 15155 log.cc:1196] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/929c78d081f84a66831be9803594fbcd
I20260812 11:16:45.108507 15156 log.cc:1196] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/929c78d081f84a66831be9803594fbcd
I20260812 11:16:45.108757 15156 ts_tablet_manager.cc:1939] T 929c78d081f84a66831be9803594fbcd P e7149eddd07948e2b0303476c6c59e1b: Deleting consensus metadata
I20260812 11:16:45.108757 15155 ts_tablet_manager.cc:1939] T 929c78d081f84a66831be9803594fbcd P 836b8cac1e3e4ab391feef2b7cbacb8f: Deleting consensus metadata
I20260812 11:16:45.108783 15154 ts_tablet_manager.cc:1918] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:16:45.108851 15154 log.cc:1196] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/929c78d081f84a66831be9803594fbcd
I20260812 11:16:45.109105 15154 ts_tablet_manager.cc:1939] T 929c78d081f84a66831be9803594fbcd P 687bc46e13ff4fac808cd77d9cb59638: Deleting consensus metadata
I20260812 11:16:45.109428 14098 catalog_manager.cc:4841] TS e7149eddd07948e2b0303476c6c59e1b (127.30.255.131:33589): tablet 929c78d081f84a66831be9803594fbcd (table test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]) successfully deleted
I20260812 11:16:45.109601 14098 catalog_manager.cc:4841] TS 836b8cac1e3e4ab391feef2b7cbacb8f (127.30.255.129:41011): tablet 929c78d081f84a66831be9803594fbcd (table test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]) successfully deleted
I20260812 11:16:45.109781 14099 catalog_manager.cc:4841] TS 687bc46e13ff4fac808cd77d9cb59638 (127.30.255.130:33217): tablet 929c78d081f84a66831be9803594fbcd (table test-table [id=5d6960690a954a1388b1a6a58ee5a7e4]) successfully deleted
I20260812 11:16:45.110150 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 14263
I20260812 11:16:45.126806 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 14520
I20260812 11:16:45.140975 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 14792
I20260812 11:16:45.152899 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 14062
2026-08-12T11:16:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3388 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:45 dist-test-slave-gjrz krb5kdc[15289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:45 dist-test-slave-gjrz krb5kdc[15289](info): set up 2 sockets
Aug 12 11:16:45 dist-test-slave-gjrz krb5kdc[15289](info): commencing operation
krb5kdc: starting...
W20260812 11:16:46.359059 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:46 dist-test-slave-gjrz krb5kdc[15289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:46.472191 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:46477
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:44387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:46477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:46.579319 15310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:46.579538 15310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:46.579576 15310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:46.583374 15310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:46.583415 15310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:46.583432 15310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:46.583446 15310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:46.583464 15310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:46.588471 15310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:44387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:46477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:46.589350 15310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:46.590266 15310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:46.598084 15320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:46.598148 15321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:46.598153 15328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:46.598202 15310 server_base.cc:1034] running on GCE node
I20260812 11:16:46.598932 15310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:46.599997 15310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:46.601154 15310 hybrid_clock.cc:648] HybridClock initialized: now 1786533406601141 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T11:16:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1457 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:16:46 dist-test-slave-gjrz krb5kdc[15336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:16:46 dist-test-slave-gjrz krb5kdc[15336](info): set up 2 sockets
Aug 12 11:16:46 dist-test-slave-gjrz krb5kdc[15336](info): commencing operation
krb5kdc: starting...
W20260812 11:16:47.811729 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:16:47 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:16:47.923470 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33989
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:33989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:48.029584 15357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:48.029806 15357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:48.029843 15357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:48.033717 15357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:48.033767 15357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:48.033785 15357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:48.033799 15357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:48.033814 15357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:48.038843 15357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:33989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:48.039773 15357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:48.040719 15357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:48.048640 15364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.048648 15365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.048784 15357 server_base.cc:1034] running on GCE node
W20260812 11:16:48.048676 15370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.049454 15357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:48.050436 15357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:48.051584 15357 hybrid_clock.cc:648] HybridClock initialized: now 1786533408051564 us; error 35 us; skew 500 ppm
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:48.054131 15357 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:48.054883 15357 webserver.cc:460] Webserver started at http://127.30.255.190:38533/ using document root <none> and password file <none>
I20260812 11:16:48.055398 15357 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:48.055461 15357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:48.055632 15357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:48.057456 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "1281d29b941c412abb251d337dbe920e"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "5a1788a75372a7077e9fa0f0ea15d841"
server_key_iv: "3305423a85f31e21493ab75e5304eb5f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.058020 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "1281d29b941c412abb251d337dbe920e"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "5a1788a75372a7077e9fa0f0ea15d841"
server_key_iv: "3305423a85f31e21493ab75e5304eb5f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.061399 15357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:16:48.063640 15382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.064798 15357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:48.064895 15357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "1281d29b941c412abb251d337dbe920e"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "5a1788a75372a7077e9fa0f0ea15d841"
server_key_iv: "3305423a85f31e21493ab75e5304eb5f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.064972 15357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:48.077844 15357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:48.080263 15357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:48.080395 15357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:48.086615 15357 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:33989
I20260812 11:16:48.086623 15472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:33989 every 8 connection(s)
I20260812 11:16:48.087483 15357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:48.088523 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 15357
I20260812 11:16:48.088647 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:16:48.088912 31742 external_mini_cluster.cc:1428] Setting key 703da28d79588d2d54b58adac03ff26b
I20260812 11:16:48.090255 15475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:16:48.096629 15475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Bootstrap starting.
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:48.098822 15475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Neither blocks nor log segments found. Creating new log.
I20260812 11:16:48.099710 15475 log.cc:824] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:48.101827 15475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: No bootstrap required, opened a new log
I20260812 11:16:48.102883 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.090180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.30.255.190:33989)
0812 11:16:48.090576 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:48.090594 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:48.090635 (+ 41us) server_negotiation.cc:407] Connection header received
0812 11:16:48.091437 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:48.091460 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:48.091850 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:48.092222 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:48.092828 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.094323 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.095115 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.095824 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.095987 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.098129 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:48.098135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:48.098149 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:48.098179 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:48.100008 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.100489 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.100495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.100501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.100635 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.100874 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.100877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.100879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.101371 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:48.101553 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:48.101740 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 11:16:48.102032 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":127,"threads_started":1}
I20260812 11:16:48.105455 15475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:48.105654 15475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:16:48.105695 15475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1281d29b941c412abb251d337dbe920e, State: Initialized, Role: FOLLOWER
I20260812 11:16:48.106040 15475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:48.106148 15475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:48.106190 15475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:48.106261 15475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:16:48.107003 15475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:48.107306 15475 leader_election.cc:302] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1281d29b941c412abb251d337dbe920e; no voters:
I20260812 11:16:48.107561 15475 leader_election.cc:288] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:16:48.107704 15483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:16:48.108024 15483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 LEADER]: Becoming Leader. State: Replica: 1281d29b941c412abb251d337dbe920e, State: Running, Role: LEADER
I20260812 11:16:48.108361 15483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:48.108697 15475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:48.109874 15486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1281d29b941c412abb251d337dbe920e. Latest consensus state: current_term: 1 leader_uuid: "1281d29b941c412abb251d337dbe920e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } } }
I20260812 11:16:48.109975 15486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:48.109926 15485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1281d29b941c412abb251d337dbe920e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } } }
I20260812 11:16:48.110026 15485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:48.110354 15501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:48.113595 15501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:48.118778 15501 catalog_manager.cc:1347] Generated new cluster ID: 4b2dbbf3d8b345dbb8a92b2276ca1700
I20260812 11:16:48.118839 15501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:48.134012 15501 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:16:48.134778 15501 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:48.139307 15501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Generated new TSK 0
I20260812 11:16:48.139883 15501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:16:48.168378 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:48.274955 15524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:48.275188 15524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:48.275245 15524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:48.279099 15524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:48.279157 15524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:48.279234 15524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:48.284241 15524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:48.285156 15524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:48.286069 15524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:48.294943 15544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.294955 15537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.295186 15539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.295118 15524 server_base.cc:1034] running on GCE node
I20260812 11:16:48.295614 15524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:48.296168 15524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:48.297338 15524 hybrid_clock.cc:648] HybridClock initialized: now 1786533408297320 us; error 28 us; skew 500 ppm
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:48.299973 15524 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:16:48.300734 15524 webserver.cc:460] Webserver started at http://127.30.255.129:34681/ using document root <none> and password file <none>
I20260812 11:16:48.301210 15524 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:48.301265 15524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:48.301424 15524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:48.303310 15524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "b64d4b6eca6040eba50218ff23f5655b"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "0ec4af3da2edbf3ba3e15533d5d8bb5c"
server_key_iv: "5667fdfc99e97a41edce8124f60b9b23"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.303831 15524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "b64d4b6eca6040eba50218ff23f5655b"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "0ec4af3da2edbf3ba3e15533d5d8bb5c"
server_key_iv: "5667fdfc99e97a41edce8124f60b9b23"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.307246 15524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:16:48.309513 15556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.310714 15524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 11:16:48.310829 15524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "b64d4b6eca6040eba50218ff23f5655b"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "0ec4af3da2edbf3ba3e15533d5d8bb5c"
server_key_iv: "5667fdfc99e97a41edce8124f60b9b23"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.310914 15524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:48.326934 15524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:48.329733 15524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:48.329891 15524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:48.330421 15524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:48.331174 15524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:48.331223 15524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.331291 15524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:48.331323 15524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.340036 15524 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:33811
I20260812 11:16:48.340095 15757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:33811 every 8 connection(s)
I20260812 11:16:48.340864 15524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:48.343539 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 15524
I20260812 11:16:48.343667 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:16:48.344051 31742 external_mini_cluster.cc:1428] Setting key 24ee851788c7951189cb7f19fff29176
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:48.354280 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.342528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:33175 (local address 127.30.255.190:33989)
0812 11:16:48.342669 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:48.342673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:48.343751 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 11:16:48.344800 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:48.344802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:48.344848 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:48.344913 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:48.346318 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.346843 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.348219 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.348552 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.348660 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.350859 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:48.350863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:48.350866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:48.350895 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:48.352402 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.352854 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.352858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.352860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.352953 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.353238 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.353242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.353243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.353459 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:48.353566 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:48.353982 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 11:16:48.354117 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:16:48.354955 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.342862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:33175)
0812 11:16:48.343591 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:48.343652 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:48.344608 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:48.345105 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:48.345114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:48.345537 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:48.346164 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.346184 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.346964 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.346968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.348094 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.348105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.348682 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.348685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.348822 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.349327 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:48.349343 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:48.350692 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:48.352511 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.352515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.352526 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.352757 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.353033 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.353035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.353036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.353154 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.353548 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:48.353552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:48.353758 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 11:16:48.353909 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 11:16:48.354136 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":111,"threads_started":1}
I20260812 11:16:48.356115 15758 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33989
I20260812 11:16:48.356310 15758 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:48.356738 15758 heartbeater.cc:511] Master 127.30.255.190:33989 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:16:48.358189 15407 ts_manager.cc:194] Registered new tserver with Master: b64d4b6eca6040eba50218ff23f5655b (127.30.255.129:33811)
I20260812 11:16:48.359313 15407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:33175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:16:48.371174 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:48.476470 15768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:48.476684 15768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:48.476727 15768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:48.480569 15768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:48.480614 15768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:48.480682 15768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:48.485576 15768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:48.486424 15768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:48.487319 15768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:48.495951 15774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.496029 15768 server_base.cc:1034] running on GCE node
W20260812 11:16:48.496028 15775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.495931 15778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.496589 15768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:48.497151 15768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:48.498306 15768 hybrid_clock.cc:648] HybridClock initialized: now 1786533408498293 us; error 28 us; skew 500 ppm
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:48.500880 15768 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:16:48.501639 15768 webserver.cc:460] Webserver started at http://127.30.255.130:44137/ using document root <none> and password file <none>
I20260812 11:16:48.502137 15768 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:48.502202 15768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:48.502378 15768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:48.504245 15768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "ec4907c852734a32897d0b5883f24ded"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "74c80069b1c90a98f1603c45c6cbd432"
server_key_iv: "528f4d3deb55b3ef8f1df5b12197b57f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.504755 15768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "ec4907c852734a32897d0b5883f24ded"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "74c80069b1c90a98f1603c45c6cbd432"
server_key_iv: "528f4d3deb55b3ef8f1df5b12197b57f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.508090 15768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:48.510380 15792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.511488 15768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:48.511595 15768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "ec4907c852734a32897d0b5883f24ded"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "74c80069b1c90a98f1603c45c6cbd432"
server_key_iv: "528f4d3deb55b3ef8f1df5b12197b57f"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.511682 15768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:48.528570 15768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:48.531044 15768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:48.531208 15768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:48.531733 15768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:48.532464 15768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:48.532509 15768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.532548 15768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:48.532577 15768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.540477 15768 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:45427
I20260812 11:16:48.540506 15979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:45427 every 8 connection(s)
I20260812 11:16:48.541256 15768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:48.546319 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 15768
I20260812 11:16:48.546440 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:16:48.546718 31742 external_mini_cluster.cc:1428] Setting key 5ee22a439be320b2db4a166fece1fe18
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:48.553918 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.542838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:59031 (local address 127.30.255.190:33989)
0812 11:16:48.542994 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:48.542999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:48.543957 (+ 958us) server_negotiation.cc:407] Connection header received
0812 11:16:48.544726 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:48.544728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:48.544774 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:48.544843 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:48.545854 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.546370 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.547658 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.547918 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.548021 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.550540 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:48.550543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:48.550545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:48.550568 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:48.551894 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.552432 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.552434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.552436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.552506 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.552855 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.552857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.552858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.553013 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:48.553082 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:48.553669 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 11:16:48.553765 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:16:48.554780 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.543184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:59031)
0812 11:16:48.543826 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:48.543864 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:48.544567 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:48.544984 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:48.544991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:48.545315 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:48.545747 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.545760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.546493 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.546496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.547539 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.547550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.548068 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.548072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.548262 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.548786 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:48.548805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:48.550374 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:48.552007 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.552012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.552025 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.552346 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.552581 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.552584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.552585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.552765 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.553121 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:48.553128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:48.553420 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 11:16:48.553631 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 11:16:48.553929 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":109,"threads_started":1}
I20260812 11:16:48.555887 15984 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33989
I20260812 11:16:48.556130 15984 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:48.556672 15984 heartbeater.cc:511] Master 127.30.255.190:33989 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:16:48.557751 15407 ts_manager.cc:194] Registered new tserver with Master: ec4907c852734a32897d0b5883f24ded (127.30.255.130:45427)
I20260812 11:16:48.558353 15407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:59031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:16:48.573583 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:48.679611 15990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:48.679826 15990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:48.679880 15990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:48.683763 15990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:48.683813 15990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:48.683887 15990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:48.688807 15990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:48.689651 15990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:48.690511 15990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:48.699388 16007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.699379 16004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:48.699379 16002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:48.699831 15990 server_base.cc:1034] running on GCE node
I20260812 11:16:48.700220 15990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:48.700763 15990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:48.701920 15990 hybrid_clock.cc:648] HybridClock initialized: now 1786533408701902 us; error 28 us; skew 500 ppm
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:48.704417 15990 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:16:48.705145 15990 webserver.cc:460] Webserver started at http://127.30.255.131:43949/ using document root <none> and password file <none>
I20260812 11:16:48.705612 15990 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:48.705667 15990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:48.705813 15990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:16:48.707643 15990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "0978538baf364e1bb03e16ab218a5264"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "c87df9f71b3f8fc9c71e242daabca8fe"
server_key_iv: "30cc9f38dfc3520bbf6702a1793e66e3"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.708114 15990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "0978538baf364e1bb03e16ab218a5264"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "c87df9f71b3f8fc9c71e242daabca8fe"
server_key_iv: "30cc9f38dfc3520bbf6702a1793e66e3"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.711519 15990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:16:48.713646 16016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.714777 15990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:48.714902 15990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "0978538baf364e1bb03e16ab218a5264"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "c87df9f71b3f8fc9c71e242daabca8fe"
server_key_iv: "30cc9f38dfc3520bbf6702a1793e66e3"
server_key_version: "encryptionkey@0"
I20260812 11:16:48.714988 15990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:48.735333 15990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:48.737742 15990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:48.737880 15990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:48.738353 15990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:48.739081 15990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:48.739131 15990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.739182 15990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:48.739212 15990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:48.747500 15990 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:42333
I20260812 11:16:48.747524 16217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:42333 every 8 connection(s)
I20260812 11:16:48.748277 15990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:48.748605 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 15990
I20260812 11:16:48.748733 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:16:48.749040 31742 external_mini_cluster.cc:1428] Setting key e257d3dd3115a5e3ed340e07809682d4
Aug 12 11:16:48 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533408, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:16:48.760361 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.750020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:47697 (local address 127.30.255.190:33989)
0812 11:16:48.750152 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:48.750156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:48.751309 (+ 1153us) server_negotiation.cc:407] Connection header received
0812 11:16:48.752132 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:48.752135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:48.752179 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:48.752242 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:48.753289 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.753808 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.754850 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.755052 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.755134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.757377 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:48.757381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:48.757384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:48.757409 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:48.758484 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.758940 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.758944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.758946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.759025 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:48.759350 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:48.759354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:48.759356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:48.759534 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:48.759627 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:48.760091 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 11:16:48.760207 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:16:48.761044 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:48.750479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:47697)
0812 11:16:48.751146 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:48.751196 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:48.751971 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:48.752384 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:48.752391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:48.752723 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:48.753175 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.753189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.753892 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.753894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.754755 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:48.754765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:48.755228 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:48.755233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:48.755422 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:48.755872 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:48.755887 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:48.757223 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:16:48.758592 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.758598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.758612 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.758847 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.759109 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:16:48.759112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:16:48.759114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:16:48.759244 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:16:48.759640 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:16:48.759647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:16:48.759863 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 11:16:48.760013 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 11:16:48.760314 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":106,"threads_started":1}
I20260812 11:16:48.762074 16218 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33989
I20260812 11:16:48.762297 16218 heartbeater.cc:464] Registering TS with master...
I20260812 11:16:48.762787 16218 heartbeater.cc:511] Master 127.30.255.190:33989 requested a full tablet report, sending...
I20260812 11:16:48.763747 15407 ts_manager.cc:194] Registered new tserver with Master: 0978538baf364e1bb03e16ab218a5264 (127.30.255.131:42333)
I20260812 11:16:48.764283 15407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:47697
I20260812 11:16:48.774092 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
Entry for principal oryx/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab.
WARNING: no policy specified for oryx/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
Entry for principal oryx/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab.
WARNING: no policy specified for oryx/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
Entry for principal oryx/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab.
I20260812 11:16:48.837183 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 15524
I20260812 11:16:48.854662 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 15768
I20260812 11:16:48.863421 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 15990
I20260812 11:16:48.870864 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 15357
I20260812 11:16:48.881902 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33989
--webserver_interface=127.30.255.190
--webserver_port=38533
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:33989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:16:48.987131 16237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:48.987373 16237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:48.987412 16237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:48.991215 16237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:16:48.991278 16237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:48.991297 16237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:48.991315 16237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:16:48.991345 16237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:16:48.996277 16237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:33989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:48.997117 16237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:48.997983 16237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:49.005707 16250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:49.005786 16237 server_base.cc:1034] running on GCE node
W20260812 11:16:49.005805 16247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.005725 16254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:49.006435 16237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:49.007440 16237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:49.008627 16237 hybrid_clock.cc:648] HybridClock initialized: now 1786533409008609 us; error 25 us; skew 500 ppm
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533409, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:49.011233 16237 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:16:49.012053 16237 webserver.cc:460] Webserver started at http://127.30.255.190:38533/ using document root <none> and password file <none>
I20260812 11:16:49.012558 16237 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:49.012637 16237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:49.016201 16237 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:49.018199 16265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.019384 16237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:49.019496 16237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "1281d29b941c412abb251d337dbe920e"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "5a1788a75372a7077e9fa0f0ea15d841"
server_key_iv: "3305423a85f31e21493ab75e5304eb5f"
server_key_version: "encryptionkey@0"
I20260812 11:16:49.019984 16237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:49.031240 16237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:49.033676 16237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:49.033880 16237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:49.040606 16237 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:33989
I20260812 11:16:49.040653 16352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:33989 every 8 connection(s)
I20260812 11:16:49.041601 16237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:16:49.045590 16355 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:16:49.047118 16355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Bootstrap starting.
I20260812 11:16:49.047441 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 16237
I20260812 11:16:49.047937 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:33811
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:33989
--webserver_port=34681
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:16:49.064213 16355 log.cc:824] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:16:49.070308 16355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:16:49.070617 16355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Bootstrap complete.
I20260812 11:16:49.074519 16355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:49.074888 16355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1281d29b941c412abb251d337dbe920e, State: Initialized, Role: FOLLOWER
I20260812 11:16:49.075385 16355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:49.075510 16355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:16:49.075577 16355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:16:49.075675 16355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:16:49.077443 16355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:49.077746 16355 leader_election.cc:302] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1281d29b941c412abb251d337dbe920e; no voters:
I20260812 11:16:49.078035 16355 leader_election.cc:288] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:16:49.078152 16365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:16:49.078329 16365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [term 2 LEADER]: Becoming Leader. State: Replica: 1281d29b941c412abb251d337dbe920e, State: Running, Role: LEADER
I20260812 11:16:49.078614 16365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } }
I20260812 11:16:49.079182 16355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:16:49.079546 16367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1281d29b941c412abb251d337dbe920e. Latest consensus state: current_term: 2 leader_uuid: "1281d29b941c412abb251d337dbe920e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } } }
I20260812 11:16:49.079666 16367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:49.079892 16366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1281d29b941c412abb251d337dbe920e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281d29b941c412abb251d337dbe920e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33989 } } }
I20260812 11:16:49.080001 16366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e [sys.catalog]: This master's current role is: LEADER
I20260812 11:16:49.080054 16373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:16:49.083124 16373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:16:49.083767 16373 catalog_manager.cc:1259] Loaded cluster ID: 4b2dbbf3d8b345dbb8a92b2276ca1700
I20260812 11:16:49.083825 16373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:16:49.086071 16373 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:16:49.086717 16373 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1281d29b941c412abb251d337dbe920e: Loaded TSK: 0
I20260812 11:16:49.087949 16373 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:16:49.157261 16361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:49.157483 16361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:49.157538 16361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:49.161454 16361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:49.161504 16361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:49.161581 16361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:16:49.166554 16361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.129.keytab
--principal=oryx/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:33811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=34681
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:49.167464 16361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:49.168370 16361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:49.177299 16407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.177314 16412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.177273 16408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:49.177886 16361 server_base.cc:1034] running on GCE node
I20260812 11:16:49.178277 16361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:49.178886 16361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:49.180040 16361 hybrid_clock.cc:648] HybridClock initialized: now 1786533409180008 us; error 49 us; skew 500 ppm
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533409, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:49.182864 16361 init.cc:377] Logged in from keytab as oryx/127.30.255.129@KRBTEST.COM (short username oryx)
I20260812 11:16:49.183748 16361 webserver.cc:460] Webserver started at http://127.30.255.129:34681/ using document root <none> and password file <none>
I20260812 11:16:49.184284 16361 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:49.184363 16361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:49.188103 16361 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:49.190147 16425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.191424 16361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:49.191524 16361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "b64d4b6eca6040eba50218ff23f5655b"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "0ec4af3da2edbf3ba3e15533d5d8bb5c"
server_key_iv: "5667fdfc99e97a41edce8124f60b9b23"
server_key_version: "encryptionkey@0"
I20260812 11:16:49.192022 16361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:49.204387 16361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:49.206975 16361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:49.207129 16361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:49.207696 16361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:49.208446 16361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:49.208493 16361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.208549 16361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:49.208576 16361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.217285 16361 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:33811
I20260812 11:16:49.217315 16627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:33811 every 8 connection(s)
I20260812 11:16:49.218164 16361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:16:49.224861 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 16361
I20260812 11:16:49.225272 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:45427
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:33989
--webserver_port=44137
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.230427 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.220100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:60633)
0812 11:16:49.220740 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.220777 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.221511 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.222559 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.222568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.222986 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.223446 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.223461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.225016 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.225019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.225932 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.225943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.226846 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.226849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.226977 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.227511 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.227526 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.229747 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":124,"threads_started":1}
W20260812 11:16:49.230890 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.219983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:60633 (local address 127.30.255.190:33989)
0812 11:16:49.220448 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.220468 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.220884 (+ 416us) server_negotiation.cc:407] Connection header received
0812 11:16:49.221873 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.221895 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.222172 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.222469 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.223625 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.224591 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.226055 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.226715 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.226876 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.230026 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:60633: BlockingRecv error: recv got EOF from 127.30.255.129:60633 (error 108)
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
W20260812 11:16:49.230958 16629 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:33989 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.255.190:33989: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.236224 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.231349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:35075)
0812 11:16:49.231433 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.231446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.231513 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.231826 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.231828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.231942 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.232040 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.232046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.232967 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.232969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.233473 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.233479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.233984 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.233986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.234048 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.234465 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.234475 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.236144 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 11:16:49.236423 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.231506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:35075 (local address 127.30.255.190:33989)
0812 11:16:49.231622 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.231626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.231640 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:49.231676 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.231679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.231724 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.231812 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.232144 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.232873 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.233573 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.233887 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.233994 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.236304 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:35075: BlockingRecv error: recv got EOF from 127.30.255.129:35075 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.241228 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.236650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:53425)
0812 11:16:49.236752 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.236765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.236835 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.237087 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.237089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.237196 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.237258 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.237262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.238186 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.238187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.238668 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.238674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.239158 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.239160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.239209 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.239613 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.239623 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.241153 (+ 1530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 11:16:49.241389 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.236715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:53425 (local address 127.30.255.190:33989)
0812 11:16:49.236815 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.236819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.236833 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:49.236953 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.236956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.237001 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.237084 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.237352 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.238101 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.238752 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.239075 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.239164 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.241290 (+ 2126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:53425: BlockingRecv error: recv got EOF from 127.30.255.129:53425 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 11:16:49.241391 16629 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:16:49.335309 16634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:49.335525 16634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:49.335563 16634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:49.339375 16634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:49.339416 16634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:49.339468 16634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:16:49.344504 16634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.130.keytab
--principal=oryx/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=44137
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:49.345353 16634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:49.346222 16634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:16:49.354974 16655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.354992 16647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.354951 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:49.355247 16634 server_base.cc:1034] running on GCE node
I20260812 11:16:49.355639 16634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:49.356184 16634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:49.357333 16634 hybrid_clock.cc:648] HybridClock initialized: now 1786533409357306 us; error 40 us; skew 500 ppm
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533409, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:49.360077 16634 init.cc:377] Logged in from keytab as oryx/127.30.255.130@KRBTEST.COM (short username oryx)
I20260812 11:16:49.360872 16634 webserver.cc:460] Webserver started at http://127.30.255.130:44137/ using document root <none> and password file <none>
I20260812 11:16:49.361359 16634 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:49.361440 16634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:49.364970 16634 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:16:49.366780 16667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.367887 16634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:49.367985 16634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "ec4907c852734a32897d0b5883f24ded"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "74c80069b1c90a98f1603c45c6cbd432"
server_key_iv: "528f4d3deb55b3ef8f1df5b12197b57f"
server_key_version: "encryptionkey@0"
I20260812 11:16:49.368456 16634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:49.387689 16634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:49.390075 16634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:49.390225 16634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:49.390731 16634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:49.391482 16634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:49.391528 16634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.391569 16634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:49.391598 16634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.399631 16634 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:45427
I20260812 11:16:49.399662 16889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:45427 every 8 connection(s)
I20260812 11:16:49.400473 16634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:16:49.402251 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 16634
I20260812 11:16:49.402704 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:42333
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:33989
--webserver_port=43949
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:16:49.410815 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.402010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:60105 (local address 127.30.255.190:33989)
0812 11:16:49.402171 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.402176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.403145 (+ 969us) server_negotiation.cc:407] Connection header received
0812 11:16:49.403973 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.403977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.404037 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.404134 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.405526 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.406163 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.407306 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.407531 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.407622 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.410696 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:60105: BlockingRecv error: recv got EOF from 127.30.255.130:60105 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:16:49.411372 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.402335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:60105)
0812 11:16:49.402996 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.403045 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.403786 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.404302 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.404311 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.404738 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.405386 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.405405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.406296 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.406299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.407156 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.407165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.407666 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.407670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.407835 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.408300 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.408316 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.410555 (+ 2239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":109,"threads_started":1}
W20260812 11:16:49.411852 16894 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:33989 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.255.190:33989: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.416759 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.412237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:39669)
0812 11:16:49.412483 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.412496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.412571 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.412782 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.412784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.412922 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.413012 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.413017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.413676 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.413679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.414196 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.414203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.414596 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.414599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.414684 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.415045 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.415059 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.416657 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163}
W20260812 11:16:49.416986 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.412349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:39669 (local address 127.30.255.190:33989)
0812 11:16:49.412444 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.412448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.412535 (+ 87us) server_negotiation.cc:407] Connection header received
0812 11:16:49.412654 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.412657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.412695 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.412756 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.413118 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.413582 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.414299 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.414504 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.414588 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.416787 (+ 2199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:39669: BlockingRecv error: recv got EOF from 127.30.255.130:39669 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.421864 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.417452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:51577)
0812 11:16:49.417552 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.417566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.417636 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.417831 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.417833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.417987 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.418051 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.418056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.418708 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.418711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.419202 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.419209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.419793 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.419796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.419851 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.420233 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.420242 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.421789 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 11:16:49.422114 16894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:16:49.422150 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.417518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:51577 (local address 127.30.255.190:33989)
0812 11:16:49.417613 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.417617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.417628 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:16:49.417707 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.417710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.417747 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.417807 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.418152 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.418617 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.419353 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.419697 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.419801 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.422035 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:51577: BlockingRecv error: recv got EOF from 127.30.255.130:51577 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 11:16:49.512962 16897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:16:49.513170 16897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:16:49.513208 16897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:16:49.517148 16897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:16:49.517195 16897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:16:49.517249 16897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:16:49.522159 16897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/krb5kdc/oryx_127.30.255.131.keytab
--principal=oryx/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:42333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=43949
--tserver_master_addrs=127.30.255.190:33989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:16:49.522955 16897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:16:49.523898 16897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:16:49.532711 16897 server_base.cc:1034] running on GCE node
W20260812 11:16:49.532663 16907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.532768 16910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:16:49.532666 16912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:16:49.533308 16897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:16:49.533842 16897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:16:49.534996 16897 hybrid_clock.cc:648] HybridClock initialized: now 1786533409534973 us; error 33 us; skew 500 ppm
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533409, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:16:49.537530 16897 init.cc:377] Logged in from keytab as oryx/127.30.255.131@KRBTEST.COM (short username oryx)
I20260812 11:16:49.538280 16897 webserver.cc:460] Webserver started at http://127.30.255.131:43949/ using document root <none> and password file <none>
I20260812 11:16:49.538762 16897 fs_manager.cc:359] Metadata directory not provided
I20260812 11:16:49.538836 16897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:16:49.542479 16897 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:16:49.546068 16925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.547179 16897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 11:16:49.547300 16897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "0978538baf364e1bb03e16ab218a5264"
format_stamp: "Formatted at 2026-08-12 11:16:48 on dist-test-slave-gjrz"
server_key: "c87df9f71b3f8fc9c71e242daabca8fe"
server_key_iv: "30cc9f38dfc3520bbf6702a1793e66e3"
server_key_version: "encryptionkey@0"
I20260812 11:16:49.547789 16897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:16:49.588405 16897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:16:49.590884 16897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:16:49.591022 16897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:16:49.591562 16897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:16:49.592300 16897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:16:49.592345 16897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.592381 16897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:16:49.592401 16897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:16:49.600907 16897 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:42333
I20260812 11:16:49.600931 17121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:42333 every 8 connection(s)
I20260812 11:16:49.601861 16897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:16:49.609859 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 16897
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:16:49.611939 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.603525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:59019 (local address 127.30.255.190:33989)
0812 11:16:49.603650 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.603654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.604612 (+ 958us) server_negotiation.cc:407] Connection header received
0812 11:16:49.605437 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.605440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.605490 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.605565 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.606629 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.607199 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.608305 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.608521 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.608593 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.611843 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:59019: BlockingRecv error: recv got EOF from 127.30.255.131:59019 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:16:49.612527 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.603888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:59019)
0812 11:16:49.604480 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.604517 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.605228 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.605711 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.605718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.606032 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.606473 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.606486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.607353 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.607356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.608213 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.608223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.608626 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.608629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.608757 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.609222 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.609238 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.611757 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":110,"threads_started":1}
W20260812 11:16:49.612980 17122 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:33989 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.255.190:33989: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.618347 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.613338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:52015)
0812 11:16:49.613455 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.613469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.613543 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.613773 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.613775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.613891 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.613991 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.613997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.614663 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.614666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.615486 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.615496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.615999 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.616002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.616072 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.616535 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.616546 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.618265 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 11:16:49.618505 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.613449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:52015 (local address 127.30.255.190:33989)
0812 11:16:49.613552 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.613556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.613567 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:16:49.613635 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.613638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.613681 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.613752 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.614097 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.614561 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.615581 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.615882 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.616000 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.618397 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:52015: BlockingRecv error: recv got EOF from 127.30.255.131:52015 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:16:49.619580 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.610802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.30.255.190:33989)
0812 11:16:49.611040 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.611043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.611054 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:16:49.611083 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.611085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.611121 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.611188 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.612137 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.612657 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.613643 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.613927 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.613999 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.615454 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:16:49.615460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:16:49.615473 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:16:49.615499 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:16:49.617165 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:49.617646 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:49.617653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:49.617661 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:49.617811 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:16:49.618089 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:16:49.618091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:16:49.618093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:16:49.618493 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:16:49.618825 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 11:16:49.619176 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 11:16:49.619421 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:49 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:49.623335 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:49.619008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:38905)
0812 11:16:49.619154 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:49.619166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:49.619247 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:49.619471 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:49.619474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:49.619615 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:49.619701 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.619708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.620331 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.620334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.620892 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:49.620899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.621321 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.621323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:49.621394 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.621781 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:49.621795 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:49.623223 (+ 1428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 11:16:49.623461 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:49.619046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:38905 (local address 127.30.255.190:33989)
0812 11:16:49.619155 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:49.619159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:49.619230 (+ 71us) server_negotiation.cc:407] Connection header received
0812 11:16:49.619334 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:49.619336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:49.619375 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:49.619432 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:49.619793 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.620235 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.621011 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:49.621225 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:49.621320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:49.623356 (+ 2036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:38905: BlockingRecv error: recv got EOF from 127.30.255.131:38905 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 11:16:49.623538 17122 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:50.246902 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:50.241897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:49419)
0812 11:16:50.242150 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:50.242164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:50.242243 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:50.242456 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:50.242458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:50.242621 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:50.242736 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.242741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.243496 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.243499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.244093 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.244103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.244476 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.244480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.244549 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.245043 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:50.245055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:50.246794 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
W20260812 11:16:50.247074 17136 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:50.241966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:49419 (local address 127.30.255.190:33989)
0812 11:16:50.242218 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:50.242222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:50.242235 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:50.242314 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:50.242317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:50.242359 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:50.242428 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:50.242820 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.243375 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.244189 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.244399 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.244462 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.246929 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:49419: BlockingRecv error: recv got EOF from 127.30.255.129:49419 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:50.427709 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:50.422636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:44723)
0812 11:16:50.422873 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:50.422886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:50.422973 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:50.423180 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:50.423182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:50.423379 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:50.423492 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.423498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.424311 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.424314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.424870 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.424878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.425234 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.425236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.425301 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.425788 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:50.425800 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:50.427570 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
W20260812 11:16:50.427810 17136 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:50.422701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:44723 (local address 127.30.255.190:33989)
0812 11:16:50.422828 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:50.422832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:50.422943 (+ 111us) server_negotiation.cc:407] Connection header received
0812 11:16:50.423042 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:50.423044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:50.423088 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:50.423154 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:50.423588 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.424219 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.424953 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.425162 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.425227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.427710 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:44723: BlockingRecv error: recv got EOF from 127.30.255.130:44723 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:50 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:50.628965 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:50.624049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:39591)
0812 11:16:50.624294 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:50.624306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:50.624396 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:50.624611 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:50.624613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:50.624777 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:50.624889 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.624894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.625554 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.625562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.626118 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:50.626125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.626471 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.626473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:50.626544 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.627008 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:50.627020 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:50.628826 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":64,"threads_started":1}
W20260812 11:16:50.629087 17136 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:50.624096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:39591 (local address 127.30.255.190:33989)
0812 11:16:50.624242 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:50.624247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:50.624362 (+ 115us) server_negotiation.cc:407] Connection header received
0812 11:16:50.624474 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:50.624476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:50.624523 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:50.624586 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:50.624968 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.625480 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.626196 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:50.626397 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:50.626464 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:50.628966 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:39591: BlockingRecv error: recv got EOF from 127.30.255.131:39591 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:51.252803 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:51.247630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:55147)
0812 11:16:51.247898 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:51.247915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:51.248017 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:51.248251 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:51.248254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:51.248441 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:51.248543 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.248549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.249231 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.249234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.249794 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.249801 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.250286 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.250289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.250353 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.250857 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:51.250869 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:51.252692 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
W20260812 11:16:51.252964 17144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:51.247703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:55147 (local address 127.30.255.190:33989)
0812 11:16:51.247931 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:51.247934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:51.247981 (+ 47us) server_negotiation.cc:407] Connection header received
0812 11:16:51.248101 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:51.248103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:51.248151 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:51.248217 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:51.248636 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.249153 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.249927 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.250178 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.250261 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.252831 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:55147: BlockingRecv error: recv got EOF from 127.30.255.129:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":59,"threads_started":1}
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:51.434765 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:51.428370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:35791)
0812 11:16:51.428646 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:51.428659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:51.428756 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:51.428953 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:51.428955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:51.429161 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:51.429284 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.429292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.430364 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.430368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.431239 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.431251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.431964 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.431968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.432052 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.432602 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:51.432617 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:51.434615 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
W20260812 11:16:51.435149 17144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:51.428439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:35791 (local address 127.30.255.190:33989)
0812 11:16:51.428583 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:51.428587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:51.428717 (+ 130us) server_negotiation.cc:407] Connection header received
0812 11:16:51.428819 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:51.428821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:51.428864 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:51.428936 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:51.429410 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.430212 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.431401 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.431745 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.431834 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.435040 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:35791: BlockingRecv error: recv got EOF from 127.30.255.130:35791 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:51 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:51.638067 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:51.629639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:33119)
0812 11:16:51.629895 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:51.629911 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:51.630016 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:51.630257 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:51.630260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:51.630447 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:51.630541 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.630547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.631317 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.631321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.631881 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:51.631890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.632322 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.632326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:51.632409 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.636123 (+ 3714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:51.636135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:51.637923 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
W20260812 11:16:51.638200 17144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:51.629755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:33119 (local address 127.30.255.190:33989)
0812 11:16:51.629885 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:51.629890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:51.629970 (+ 80us) server_negotiation.cc:407] Connection header received
0812 11:16:51.630104 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:51.630108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:51.630153 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:51.630216 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:51.630659 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.631189 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.631992 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:51.632220 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:51.632308 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:51.638072 (+ 5764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:33119: BlockingRecv error: recv got EOF from 127.30.255.131:33119 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:52.258492 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:52.253494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:44513)
0812 11:16:52.253750 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:52.253763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:52.253857 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:52.254085 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:52.254088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:52.254306 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:52.254430 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.254437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.255158 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.255161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.255757 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.255764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.256127 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.256129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.256186 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.256661 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:52.256673 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:52.258369 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
W20260812 11:16:52.258635 17149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:52.253564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:44513 (local address 127.30.255.190:33989)
0812 11:16:52.253827 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:52.253831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:52.253844 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:52.253938 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:52.253940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:52.253982 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:52.254056 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:52.254534 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.255071 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.255842 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.256053 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.256126 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.258504 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:44513: BlockingRecv error: recv got EOF from 127.30.255.129:44513 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":112,"threads_started":1}
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:52.441540 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:52.436442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:53107)
0812 11:16:52.436682 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:52.436695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:52.436790 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:52.436993 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:52.436995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:52.437170 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:52.437268 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.437273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.437935 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.437937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.438504 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.438512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.438863 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.438865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.438922 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.439422 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:52.439435 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:52.441399 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":67,"threads_started":1}
W20260812 11:16:52.441635 17149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:52.436528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:53107 (local address 127.30.255.190:33989)
0812 11:16:52.436651 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:52.436654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:52.436753 (+ 99us) server_negotiation.cc:407] Connection header received
0812 11:16:52.436858 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:52.436860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:52.436904 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:52.436968 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:52.437352 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.437865 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.438583 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.438788 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.438855 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.441541 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:53107: BlockingRecv error: recv got EOF from 127.30.255.130:53107 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:52 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:52.644099 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:52.638867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:48859)
0812 11:16:52.639140 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:52.639154 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:52.639248 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:52.639478 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:52.639480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:52.639652 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:52.639751 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.639757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.640493 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.640496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.641149 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:52.641161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.641623 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.641627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:52.641715 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.642187 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:52.642199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:52.643963 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
W20260812 11:16:52.644193 17149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:52.638944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:48859 (local address 127.30.255.190:33989)
0812 11:16:52.639063 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:52.639068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:52.639212 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:16:52.639342 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:52.639344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:52.639389 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:52.639452 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:52.639831 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.640370 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.641267 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:52.641510 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:52.641612 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:52.644089 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:48859: BlockingRecv error: recv got EOF from 127.30.255.131:48859 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:53.264205 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:53.259202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:58323)
0812 11:16:53.259495 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:53.259509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:53.259594 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:53.259819 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:53.259822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:53.259999 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:53.260100 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.260106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.260828 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.260831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.261387 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.261394 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.261765 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.261767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.261826 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.262289 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:53.262300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:53.264073 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
W20260812 11:16:53.264377 17155 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:53.259298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:58323 (local address 127.30.255.190:33989)
0812 11:16:53.259528 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:53.259532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:53.259555 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:16:53.259665 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:53.259667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:53.259709 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:53.259782 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:53.260216 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.260747 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.261467 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.261681 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.261767 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.264240 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:58323: BlockingRecv error: recv got EOF from 127.30.255.129:58323 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:53.447183 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:53.442239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:42661)
0812 11:16:53.442523 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:53.442535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:53.442645 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:53.442846 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:53.442848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:53.443021 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:53.443121 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.443128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.443825 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.443828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.444389 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.444396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.444745 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.444747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.444806 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.445256 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:53.445267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:53.447040 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
W20260812 11:16:53.447425 17155 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:53.442323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:42661 (local address 127.30.255.190:33989)
0812 11:16:53.442447 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:53.442451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:53.442595 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:16:53.442715 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:53.442717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:53.442759 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:53.442822 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:53.443205 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.443752 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.444469 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.444670 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.444734 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.447186 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:42661: BlockingRecv error: recv got EOF from 127.30.255.130:42661 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:53 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:53.649718 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:53.644746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:53991)
0812 11:16:53.644994 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:53.645006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:53.645102 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:53.645303 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:53.645305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:53.645481 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:53.645591 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.645597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.646269 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.646271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.646823 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:53.646830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.647200 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.647203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:53.647289 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.647757 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:53.647769 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:53.649598 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
W20260812 11:16:53.649825 17155 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:53.644829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:53991 (local address 127.30.255.190:33989)
0812 11:16:53.644946 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:53.644949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:53.645057 (+ 108us) server_negotiation.cc:407] Connection header received
0812 11:16:53.645165 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:53.645167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:53.645214 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:53.645278 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:53.645673 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.646194 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.646901 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:53.647103 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:53.647194 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:53.649725 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:53991: BlockingRecv error: recv got EOF from 127.30.255.131:53991 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:54.269831 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:54.264882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:53601)
0812 11:16:54.265159 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:54.265173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:54.265260 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:54.265461 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:54.265463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:54.265636 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:54.265735 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.265741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.266486 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.266489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.267096 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.267106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.267545 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.267548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.267612 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.268055 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:54.268066 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:54.269718 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
W20260812 11:16:54.269963 17161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:54.264943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:53601 (local address 127.30.255.190:33989)
0812 11:16:54.265188 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:54.265191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:54.265219 (+ 28us) server_negotiation.cc:407] Connection header received
0812 11:16:54.265332 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:54.265334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:54.265375 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:54.265446 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:54.265843 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.266393 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.267199 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.267456 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.267533 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.269846 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:53601: BlockingRecv error: recv got EOF from 127.30.255.129:53601 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:54.453328 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:54.447887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:42339)
0812 11:16:54.448161 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:54.448176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:54.448285 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:54.448518 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:54.448521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:54.448722 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:54.448817 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.448824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.449574 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.449578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.450164 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.450173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.450671 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.450675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.450769 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.451384 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:54.451396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:54.453178 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
W20260812 11:16:54.453420 17161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:54.448029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:42339 (local address 127.30.255.190:33989)
0812 11:16:54.448171 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:54.448176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:54.448239 (+ 63us) server_negotiation.cc:407] Connection header received
0812 11:16:54.448368 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:54.448372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:54.448420 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:54.448485 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:54.448928 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.449468 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.450276 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.450546 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.450640 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.453322 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:42339: BlockingRecv error: recv got EOF from 127.30.255.130:42339 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:54 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:54.660561 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:54.650434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:44077)
0812 11:16:54.650694 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:54.650707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:54.650799 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:54.651076 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:54.651078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:54.651257 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:54.651379 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.651385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.652115 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.652119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.652710 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:54.652722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.653114 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.653116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:54.653177 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.658690 (+ 5513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:54.658703 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:54.660440 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
W20260812 11:16:54.660697 17161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:54.650505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:44077 (local address 127.30.255.190:33989)
0812 11:16:54.650627 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:54.650631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:54.650764 (+ 133us) server_negotiation.cc:407] Connection header received
0812 11:16:54.650867 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:54.650869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:54.650914 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:54.651002 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:54.651460 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.651977 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.652805 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:54.653023 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:54.653095 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:54.660575 (+ 7480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:44077: BlockingRecv error: recv got EOF from 127.30.255.131:44077 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:55.276098 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:55.270573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:35837)
0812 11:16:55.270824 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:55.270838 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:55.270928 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:55.271216 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:55.271219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:55.271490 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:55.271636 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.271646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.272653 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.272655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.273229 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.273238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.273627 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.273629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.273690 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.274153 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:55.274164 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:55.275946 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
W20260812 11:16:55.276211 17168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:55.270653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:35837 (local address 127.30.255.190:33989)
0812 11:16:55.270874 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:55.270878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:55.270892 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:16:55.271007 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:55.271010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:55.271066 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:55.271175 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:55.271767 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.272542 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.273319 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.273543 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.273619 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.276087 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:35837: BlockingRecv error: recv got EOF from 127.30.255.129:35837 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:55.459401 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:55.454246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:37593)
0812 11:16:55.454535 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:55.454548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:55.454659 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:55.454902 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:55.454905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:55.455125 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:55.455226 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.455234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.456007 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.456010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.456575 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.456583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.457010 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.457013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.457109 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.457548 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:55.457561 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:55.459231 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":93,"threads_started":1}
W20260812 11:16:55.459506 17168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:55.454327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37593 (local address 127.30.255.190:33989)
0812 11:16:55.454465 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:55.454470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:55.454617 (+ 147us) server_negotiation.cc:407] Connection header received
0812 11:16:55.454748 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:55.454752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:55.454799 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:55.454864 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:55.455370 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.455899 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.456690 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.456909 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.456994 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.459406 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:37593: BlockingRecv error: recv got EOF from 127.30.255.130:37593 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:55 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:55.666399 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:55.661240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:55251)
0812 11:16:55.661496 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:55.661511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:55.661655 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:55.661973 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:55.661976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:55.662169 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:55.662271 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.662279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.663126 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.663129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.663724 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:55.663734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.664094 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.664096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:55.664154 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.664594 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:55.664605 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:55.666285 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":67,"threads_started":1}
W20260812 11:16:55.666525 17168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:55.661320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:55251 (local address 127.30.255.190:33989)
0812 11:16:55.661481 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:55.661485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:55.661585 (+ 100us) server_negotiation.cc:407] Connection header received
0812 11:16:55.661761 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:55.661764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:55.661821 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:55.661907 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:55.662400 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.663027 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.663807 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:55.664017 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:55.664083 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:55.666409 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:55251: BlockingRecv error: recv got EOF from 127.30.255.131:55251 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:56.282112 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:56.276883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:49777)
0812 11:16:56.277255 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:56.277269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:56.277393 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:56.277668 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:56.277670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:56.277847 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:56.277948 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.277953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.278636 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.278638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.279189 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.279196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.279633 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.279636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.279703 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.280172 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:56.280183 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:56.281998 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":76,"threads_started":1}
W20260812 11:16:56.282261 17175 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:56.276916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:49777 (local address 127.30.255.190:33989)
0812 11:16:56.277241 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:56.277245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:56.277353 (+ 108us) server_negotiation.cc:407] Connection header received
0812 11:16:56.277491 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:56.277493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:56.277551 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:56.277652 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:56.278034 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.278556 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.279286 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.279534 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.279629 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.282136 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:49777: BlockingRecv error: recv got EOF from 127.30.255.129:49777 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":84,"threads_started":1}
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:56.465056 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:56.460165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:37435)
0812 11:16:56.460460 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:56.460473 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:56.460582 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:56.460788 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:56.460790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:56.460967 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:56.461064 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.461069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.461734 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.461737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.462295 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.462303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.462660 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.462662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.462720 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.463145 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:56.463155 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:56.464905 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":71,"threads_started":1}
W20260812 11:16:56.465183 17175 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:56.460246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37435 (local address 127.30.255.190:33989)
0812 11:16:56.460373 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:56.460377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:56.460531 (+ 154us) server_negotiation.cc:407] Connection header received
0812 11:16:56.460659 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:56.460661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:56.460705 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:56.460761 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:56.461151 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.461655 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.462378 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.462583 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.462650 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.465044 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:37435: BlockingRecv error: recv got EOF from 127.30.255.130:37435 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:56 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:56.676484 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:56.667054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:39735)
0812 11:16:56.667378 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:56.667391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:56.667489 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:56.667692 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:56.667694 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:56.667879 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:56.667977 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.667982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.668651 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.668653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.669211 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:56.669219 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.669664 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.669667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:56.669734 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.674570 (+ 4836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:56.674584 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:56.676365 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":70,"threads_started":1}
W20260812 11:16:56.676631 17175 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:56.667109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:39735 (local address 127.30.255.190:33989)
0812 11:16:56.667236 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:56.667239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:56.667448 (+ 209us) server_negotiation.cc:407] Connection header received
0812 11:16:56.667552 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:56.667554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:56.667601 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:56.667663 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:56.668066 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.668576 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.669333 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:56.669566 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:56.669656 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:56.676508 (+ 6852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:39735: BlockingRecv error: recv got EOF from 127.30.255.131:39735 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:57.288044 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:57.282828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:57583)
0812 11:16:57.283103 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:57.283116 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:57.283214 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:57.283461 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:57.283463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:57.283639 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:57.283742 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.283747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.284492 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.284495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.285066 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.285074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.285534 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.285537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.285604 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.286085 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:57.286097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:57.287893 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":74,"threads_started":1}
W20260812 11:16:57.288154 17180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:57.282982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:57583 (local address 127.30.255.190:33989)
0812 11:16:57.283199 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:57.283203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:57.283215 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:16:57.283310 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:57.283312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:57.283357 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:57.283422 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:57.283828 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.284375 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.285190 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.285424 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.285506 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.288039 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:57583: BlockingRecv error: recv got EOF from 127.30.255.129:57583 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:57.471248 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:57.465827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:55551)
0812 11:16:57.466134 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:57.466147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:57.466255 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:57.466499 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:57.466502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:57.466701 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:57.466800 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.466806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.467775 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.467779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.468333 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.468341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.468776 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.468780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.468867 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.469339 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:57.469351 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:57.471114 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":105,"threads_started":1}
W20260812 11:16:57.471386 17180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:57.465914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:55551 (local address 127.30.255.190:33989)
0812 11:16:57.466055 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:57.466060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:57.466213 (+ 153us) server_negotiation.cc:407] Connection header received
0812 11:16:57.466343 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:57.466346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:57.466396 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:57.466460 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:57.466939 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.467668 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.468452 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.468675 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.468762 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.471239 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:55551: BlockingRecv error: recv got EOF from 127.30.255.130:55551 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:57 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:57.682255 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:57.677221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:45707)
0812 11:16:57.677546 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:57.677559 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:57.677651 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:57.677855 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:57.677857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:57.678032 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:57.678140 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.678146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.678798 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.678800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.679476 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:57.679488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.679896 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.679899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:57.679958 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.680410 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:57.680425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:57.682142 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
W20260812 11:16:57.682399 17180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:57.677298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45707 (local address 127.30.255.190:33989)
0812 11:16:57.677437 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:57.677441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:57.677615 (+ 174us) server_negotiation.cc:407] Connection header received
0812 11:16:57.677724 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:57.677726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:57.677774 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:57.677829 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:57.678221 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.678726 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.679585 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:57.679807 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:57.679877 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:57.682284 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:45707: BlockingRecv error: recv got EOF from 127.30.255.131:45707 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:58.293823 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:58.288827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:33735)
0812 11:16:58.289111 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:58.289125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:58.289214 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:58.289428 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:58.289430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:58.289617 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:58.289717 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.289723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.290404 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.290407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.290956 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.290963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.291351 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.291354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.291414 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.291849 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:58.291860 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:58.293675 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
W20260812 11:16:58.293926 17185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:58.288887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:33735 (local address 127.30.255.190:33989)
0812 11:16:58.289158 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:58.289162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:58.289175 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:16:58.289293 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:58.289295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:58.289339 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:58.289402 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:58.289801 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.290330 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.291036 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.291251 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.291332 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.293810 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:33735: BlockingRecv error: recv got EOF from 127.30.255.129:33735 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":101,"threads_started":1}
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:58.477429 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:58.471977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:38325)
0812 11:16:58.472261 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:58.472293 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:58.472406 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:58.472672 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:58.472675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:58.472875 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:58.472970 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.472976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.473728 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.473732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.474293 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.474302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.474809 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.474813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.474900 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.475343 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:58.475355 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:58.477288 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":118,"threads_started":1}
W20260812 11:16:58.477501 17185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:58.472067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:38325 (local address 127.30.255.190:33989)
0812 11:16:58.472212 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:58.472217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:58.472351 (+ 134us) server_negotiation.cc:407] Connection header received
0812 11:16:58.472508 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:58.472512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:58.472575 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:58.472650 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:58.473085 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.473623 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.474407 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.474700 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.474791 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.477409 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:38325: BlockingRecv error: recv got EOF from 127.30.255.130:38325 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:58 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:58.687903 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:58.682966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:53351)
0812 11:16:58.683232 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:58.683244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:58.683357 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:58.683585 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:58.683587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:58.683765 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:58.683861 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.683866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.684543 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.684545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.685100 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:58.685108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.685456 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.685459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:58.685516 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.685960 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:58.685971 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:58.687759 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
W20260812 11:16:58.688012 17185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:58.683039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:53351 (local address 127.30.255.190:33989)
0812 11:16:58.683165 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:58.683169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:58.683320 (+ 151us) server_negotiation.cc:407] Connection header received
0812 11:16:58.683439 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:58.683441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:58.683485 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:58.683550 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:58.683942 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.684466 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.685180 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:58.685380 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:58.685444 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:58.687897 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:53351: BlockingRecv error: recv got EOF from 127.30.255.131:53351 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:59.299531 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:59.294511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:60059)
0812 11:16:59.294794 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:59.294807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:59.294903 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:59.295116 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:59.295118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:59.295346 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:59.295451 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.295457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.296236 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.296238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.296818 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.296827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.297229 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.297231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.297288 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.297728 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:59.297738 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:59.299418 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
W20260812 11:16:59.299666 17194 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:59.294565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:60059 (local address 127.30.255.190:33989)
0812 11:16:59.294816 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:59.294820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:59.294860 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:16:59.294973 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:59.294975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:59.295018 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:59.295089 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:59.295545 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.296136 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.296917 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.297143 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.297221 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.299544 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:60059: BlockingRecv error: recv got EOF from 127.30.255.129:60059 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:59.483446 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:59.478102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:55955)
0812 11:16:59.478357 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:59.478370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:59.478461 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:59.478661 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:59.478663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:59.478831 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:59.478972 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.478979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.479717 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.479721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.480528 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.480540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.481002 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.481005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.481083 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.481503 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:59.481514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:59.483299 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":67,"threads_started":1}
W20260812 11:16:59.483528 17194 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:59.478170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:55955 (local address 127.30.255.190:33989)
0812 11:16:59.478310 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:59.478315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:59.478432 (+ 117us) server_negotiation.cc:407] Connection header received
0812 11:16:59.478528 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:59.478530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:59.478575 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:59.478646 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:59.479064 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.479592 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.480645 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.480914 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.480992 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.483430 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:55955: BlockingRecv error: recv got EOF from 127.30.255.130:55955 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:16:59 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:16:59.693770 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:16:59.688669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:43979)
0812 11:16:59.688922 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:16:59.688935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:16:59.689027 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:16:59.689226 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:16:59.689228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:16:59.689404 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:16:59.689500 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.689505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.690178 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.690180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.690741 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:16:59.690748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.691115 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.691118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:16:59.691174 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.691634 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:16:59.691646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:16:59.693594 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":70,"threads_started":1}
W20260812 11:16:59.693825 17194 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:16:59.688806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:43979 (local address 127.30.255.190:33989)
0812 11:16:59.688922 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:16:59.688926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:16:59.688983 (+ 57us) server_negotiation.cc:407] Connection header received
0812 11:16:59.689100 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:16:59.689102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:16:59.689145 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:16:59.689200 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:16:59.689583 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.690100 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.690826 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:16:59.691036 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:16:59.691108 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:16:59.693724 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:43979: BlockingRecv error: recv got EOF from 127.30.255.131:43979 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:00.305264 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:00.300215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:49313)
0812 11:17:00.300465 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:00.300479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:00.300564 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:00.300792 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:00.300794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:00.300974 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:00.301074 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.301079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.301767 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.301769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.302340 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.302348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.302752 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.302755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.302810 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.303257 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:00.303292 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:00.305133 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
W20260812 11:17:00.305370 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:00.300297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:49313 (local address 127.30.255.190:33989)
0812 11:17:00.300552 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:00.300555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:00.300569 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:17:00.300643 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:00.300645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:00.300688 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:00.300768 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:00.301160 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.301687 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.302433 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.302669 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.302746 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.305262 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:49313: BlockingRecv error: recv got EOF from 127.30.255.129:49313 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:00.489409 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:00.484162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:55977)
0812 11:17:00.484433 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:00.484446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:00.484549 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:00.484799 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:00.484802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:00.485011 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:00.485117 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.485124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.485840 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.485843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.486422 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.486431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.486848 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.486851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.486915 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.487413 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:00.487425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:00.489241 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
W20260812 11:17:00.489485 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:00.484250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:55977 (local address 127.30.255.190:33989)
0812 11:17:00.484401 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:00.484405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:00.484509 (+ 104us) server_negotiation.cc:407] Connection header received
0812 11:17:00.484620 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:00.484622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:00.484665 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:00.484733 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:00.485210 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.485717 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.486535 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.486757 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.486833 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.489384 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:55977: BlockingRecv error: recv got EOF from 127.30.255.130:55977 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:00 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:00.699582 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:00.694536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:55309)
0812 11:17:00.694798 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:00.694811 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:00.694909 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:00.695137 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:00.695139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:00.695353 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:00.695452 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.695459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.696190 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.696193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.696760 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:00.696768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.697152 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.697154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:00.697212 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.697646 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:00.697657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:00.699469 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
W20260812 11:17:00.699712 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:00.694601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:55309 (local address 127.30.255.190:33989)
0812 11:17:00.694757 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:00.694761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:00.694875 (+ 114us) server_negotiation.cc:407] Connection header received
0812 11:17:00.694978 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:00.694980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:00.695027 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:00.695108 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:00.695537 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.696096 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.696851 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:00.697069 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:00.697144 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:00.699596 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:55309: BlockingRecv error: recv got EOF from 127.30.255.131:55309 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:01.311193 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:01.305946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:54407)
0812 11:17:01.306227 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:01.306241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:01.306353 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:01.306615 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:01.306619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:01.306817 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:01.306917 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.306924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.307732 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.307736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.308294 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.308302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.308745 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.308749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.308838 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.309295 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:01.309306 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:01.311034 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
W20260812 11:17:01.311314 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:01.306048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:54407 (local address 127.30.255.190:33989)
0812 11:17:01.306267 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:01.306270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:01.306284 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:17:01.306444 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:01.306446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:01.306489 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:01.306581 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:01.307039 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.307628 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.308408 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.308642 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.308728 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.311172 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:54407: BlockingRecv error: recv got EOF from 127.30.255.129:54407 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":56,"threads_started":1}
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:01.496817 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:01.490307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:57549)
0812 11:17:01.490613 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:01.490630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:01.490751 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:01.491537 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:01.491540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:01.491746 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:01.491849 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.491855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.492610 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.492613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.493174 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.493182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.493626 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.493630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.493720 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.494304 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:01.494321 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:01.496660 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
W20260812 11:17:01.497129 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:01.490396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:57549 (local address 127.30.255.190:33989)
0812 11:17:01.490538 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:01.490543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:01.491332 (+ 789us) server_negotiation.cc:407] Connection header received
0812 11:17:01.491373 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:01.491376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:01.491422 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:01.491492 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:01.491966 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.492494 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.493289 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.493519 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.493610 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.496813 (+ 3203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:57549: BlockingRecv error: recv got EOF from 127.30.255.130:57549 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:01 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:01.712157 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:01.700297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:51003)
0812 11:17:01.700558 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:01.700572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:01.700683 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:01.700890 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:01.700892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:01.701105 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:01.701210 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.701217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.702124 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.702128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.702698 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:01.702708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.703170 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.703175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:01.703292 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.710280 (+ 6988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:01.710293 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:01.712002 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
W20260812 11:17:01.712258 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:01.700393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:51003 (local address 127.30.255.190:33989)
0812 11:17:01.700515 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:01.700519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:01.700633 (+ 114us) server_negotiation.cc:407] Connection header received
0812 11:17:01.700754 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:01.700756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:01.700802 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:01.700866 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:01.701330 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.701999 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.702818 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:01.703040 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:01.703141 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:01.712153 (+ 9012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:51003: BlockingRecv error: recv got EOF from 127.30.255.131:51003 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:02.317020 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:02.311929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:33751)
0812 11:17:02.312190 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:02.312203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:02.312302 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:02.312513 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:02.312515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:02.312707 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:02.312809 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.312814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.313548 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.313551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.314123 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.314130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.314504 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.314506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.314567 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.315026 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:02.315037 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:02.316871 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
W20260812 11:17:02.317138 17214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:02.311977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:33751 (local address 127.30.255.190:33989)
0812 11:17:02.312200 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:02.312204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:02.312263 (+ 59us) server_negotiation.cc:407] Connection header received
0812 11:17:02.312370 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:02.312372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:02.312416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:02.312490 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:02.312920 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.313465 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.314209 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.314424 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.314493 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.317012 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:33751: BlockingRecv error: recv got EOF from 127.30.255.129:33751 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:02.502869 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:02.497531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:37009)
0812 11:17:02.497793 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:02.497806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:02.497902 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:02.498107 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:02.498109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:02.498303 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:02.498400 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.498405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.499361 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.499365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.500093 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.500116 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.500527 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.500529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.500590 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.501033 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:02.501046 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:02.502754 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
W20260812 11:17:02.502995 17214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:02.497661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37009 (local address 127.30.255.190:33989)
0812 11:17:02.497785 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:02.497789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:02.497864 (+ 75us) server_negotiation.cc:407] Connection header received
0812 11:17:02.497973 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:02.497975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:02.498019 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:02.498085 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:02.498509 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.499205 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.500220 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.500438 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.500515 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.502877 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:37009: BlockingRecv error: recv got EOF from 127.30.255.130:37009 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:02 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:02.717952 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:02.712931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:43793)
0812 11:17:02.713253 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:02.713265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:02.713366 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:02.713579 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:02.713581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:02.713764 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:02.713861 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.713867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.714554 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.714556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.715131 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:02.715141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.715594 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.715597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:02.715666 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.716101 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:02.716112 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:02.717835 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":79,"threads_started":1}
W20260812 11:17:02.718079 17214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:02.712988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:43793 (local address 127.30.255.190:33989)
0812 11:17:02.713118 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:02.713123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:02.713338 (+ 215us) server_negotiation.cc:407] Connection header received
0812 11:17:02.713436 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:02.713439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:02.713487 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:02.713552 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:02.713945 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.714468 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.715239 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:02.715495 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:02.715569 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:02.717970 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:43793: BlockingRecv error: recv got EOF from 127.30.255.131:43793 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:03.322908 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:03.317786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:43075)
0812 11:17:03.318085 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:03.318098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:03.318189 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:03.318408 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:03.318410 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:03.318587 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:03.318698 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.318703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.319450 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.319452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.320008 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.320015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.320381 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.320384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.320441 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.320930 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:03.320943 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:03.322759 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":115,"threads_started":1}
W20260812 11:17:03.323163 17223 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:03.317863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:43075 (local address 127.30.255.190:33989)
0812 11:17:03.318163 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:03.318167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:03.318180 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:17:03.318265 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:03.318267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:03.318312 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:03.318387 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:03.318782 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.319373 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.320089 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.320306 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.320375 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.322931 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:43075: BlockingRecv error: recv got EOF from 127.30.255.129:43075 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":60,"threads_started":1}
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:03.509001 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:03.503607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:59621)
0812 11:17:03.503855 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:03.503868 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:03.503975 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:03.504213 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:03.504216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:03.504413 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:03.504507 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.504513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.505259 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.505263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.505873 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.505883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.506318 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.506322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.506434 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.506925 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:03.506941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:03.508854 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
W20260812 11:17:03.509104 17223 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:03.503650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:59621 (local address 127.30.255.190:33989)
0812 11:17:03.503787 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:03.503792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:03.503929 (+ 137us) server_negotiation.cc:407] Connection header received
0812 11:17:03.504066 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:03.504069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:03.504117 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:03.504181 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:03.504616 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.505156 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.505988 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.506217 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.506303 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.508994 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:59621: BlockingRecv error: recv got EOF from 127.30.255.130:59621 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:03 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.131@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:03.724071 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:03.718674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.131:42243)
0812 11:17:03.718932 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:03.718961 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:03.719068 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:03.719329 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:03.719333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:03.719544 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:03.719641 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.719647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.720394 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.720397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.720962 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:03.720971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.721410 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.721414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:03.721502 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.721995 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:03.722007 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:03.723913 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
W20260812 11:17:03.724157 17223 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:03.718739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:42243 (local address 127.30.255.190:33989)
0812 11:17:03.718873 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:03.718878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:03.719031 (+ 153us) server_negotiation.cc:407] Connection header received
0812 11:17:03.719152 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:03.719155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:03.719207 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:03.719319 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:03.719756 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.720288 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.721075 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:03.721308 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:03.721396 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:03.724033 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.131:42243: BlockingRecv error: recv got EOF from 127.30.255.131:42243 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 11:17:04 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:04 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.129@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:04.329272 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:04.323740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.129:36795)
0812 11:17:04.324024 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:04.324041 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:04.324176 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:04.324402 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:04.324404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:04.324582 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:04.324684 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:04.324689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.325621 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.325625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:04.326403 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:04.326414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.326826 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.326828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:04.326884 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:04.327347 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:04.327361 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:04.329125 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":107,"threads_started":1}
W20260812 11:17:04.329375 17232 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:04.323823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:36795 (local address 127.30.255.190:33989)
0812 11:17:04.324067 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:04.324071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:04.324117 (+ 46us) server_negotiation.cc:407] Connection header received
0812 11:17:04.324257 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:04.324259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:04.324304 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:04.324377 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:04.324811 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.325497 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.326515 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.326744 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.326819 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:04.329260 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.129:36795: BlockingRecv error: recv got EOF from 127.30.255.129:36795 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":66,"threads_started":1}
Aug 12 11:17:04 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:17:04 dist-test-slave-gjrz krb5kdc[15336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.255.130@KRBTEST.COM for oryx/127.30.255.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:17:04.514977 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:04.509854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33989 (local address 127.30.255.130:47963)
0812 11:17:04.510155 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:04.510167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:04.510269 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:04.510511 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:04.510513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:04.510716 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:04.510816 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:04.510822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.511617 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.511621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:04.512182 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:04.512191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.512618 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.512622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:04.512702 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:04.513166 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:04.513178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:04.514814 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.255.190:33989: Server oryx/127.30.255.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
W20260812 11:17:04.515081 17232 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:04.509950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:47963 (local address 127.30.255.190:33989)
0812 11:17:04.510103 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:04.510108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:04.510230 (+ 122us) server_negotiation.cc:407] Connection header received
0812 11:17:04.510356 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:04.510360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:04.510408 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:04.510472 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:04.510931 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.511512 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.512296 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.512517 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.512605 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:04.514956 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.255.130:47963: BlockingRecv error: recv got EOF from 127.30.255.130:47963 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:17:04.615978 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 16361
I20260812 11:17:04.629637 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 16634
I20260812 11:17:04.654627 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 16897
I20260812 11:17:04.666512 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 16237
2026-08-12T11:17:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18051 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T11:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:17:04Z Disabled control of system clock
I20260812 11:17:04.700455 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37453
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:37453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 11:17:04.811652 17237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:04.811858 17237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:04.811895 17237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:04.815704 17237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:17:04.815748 17237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:04.815761 17237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:04.815775 17237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:17:04.815788 17237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:17:04.820236 17237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:37453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:04.821055 17237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:04.822216 17237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:04.830142 17247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:04.830291 17248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:04.830137 17253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:04.830371 17237 server_base.cc:1034] running on GCE node
I20260812 11:17:04.831162 17237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:04.832448 17237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:04.833635 17237 hybrid_clock.cc:648] HybridClock initialized: now 1786533424833601 us; error 41 us; skew 500 ppm
I20260812 11:17:04.835704 17237 webserver.cc:460] Webserver started at http://127.30.255.190:37509/ using document root <none> and password file <none>
I20260812 11:17:04.836302 17237 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:04.836366 17237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:04.836536 17237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:04.838737 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "1cce48f7885d42d1bbc2479f80f5b9eb"
format_stamp: "Formatted at 2026-08-12 11:17:04 on dist-test-slave-gjrz"
server_key: "869f843ae009fbfa2be2531108958314"
server_key_iv: "66f2fbf18eba07a20c08f8b0163ed94b"
server_key_version: "encryptionkey@0"
I20260812 11:17:04.839567 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "1cce48f7885d42d1bbc2479f80f5b9eb"
format_stamp: "Formatted at 2026-08-12 11:17:04 on dist-test-slave-gjrz"
server_key: "869f843ae009fbfa2be2531108958314"
server_key_iv: "66f2fbf18eba07a20c08f8b0163ed94b"
server_key_version: "encryptionkey@0"
I20260812 11:17:04.844314 17237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:17:04.847337 17263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:04.848935 17237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:17:04.849062 17237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "1cce48f7885d42d1bbc2479f80f5b9eb"
format_stamp: "Formatted at 2026-08-12 11:17:04 on dist-test-slave-gjrz"
server_key: "869f843ae009fbfa2be2531108958314"
server_key_iv: "66f2fbf18eba07a20c08f8b0163ed94b"
server_key_version: "encryptionkey@0"
I20260812 11:17:04.849187 17237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:04.889844 17237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:04.890457 17237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:04.890617 17237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:04.897132 17237 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:37453
I20260812 11:17:04.897158 17342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:37453 every 8 connection(s)
I20260812 11:17:04.898032 17237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:17:04.901175 17344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:04.905519 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 17237
I20260812 11:17:04.905643 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:17:04.905985 31742 external_mini_cluster.cc:1428] Setting key acb5ae10ca23d1d001c8793b22bfa93e
I20260812 11:17:04.906775 17344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb: Bootstrap starting.
I20260812 11:17:04.909072 17344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:04.909759 17344 log.cc:824] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:04.911592 17344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb: No bootstrap required, opened a new log
I20260812 11:17:04.914984 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:04.907436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.30.255.190:37453)
0812 11:17:04.908047 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:04.908061 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:04.908095 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:17:04.908732 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:04.908750 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:04.909135 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:04.909454 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:04.909916 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.911026 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.911954 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:04.912419 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:04.912567 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:04.913352 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:04.913357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:04.913369 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:04.913741 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:04.913795 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:04.913815 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:04.913899 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:04.914046 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 11:17:04.914292 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":137,"threads_started":1}
I20260812 11:17:04.914922 17344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } }
I20260812 11:17:04.915155 17344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:04.915213 17344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cce48f7885d42d1bbc2479f80f5b9eb, State: Initialized, Role: FOLLOWER
I20260812 11:17:04.915661 17344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } }
I20260812 11:17:04.915802 17344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:17:04.915853 17344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:17:04.915932 17344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:04.916708 17344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } }
I20260812 11:17:04.917049 17344 leader_election.cc:302] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cce48f7885d42d1bbc2479f80f5b9eb; no voters:
I20260812 11:17:04.917341 17344 leader_election.cc:288] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:17:04.917457 17353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:04.917667 17353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [term 1 LEADER]: Becoming Leader. State: Replica: 1cce48f7885d42d1bbc2479f80f5b9eb, State: Running, Role: LEADER
I20260812 11:17:04.917951 17353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } }
I20260812 11:17:04.918390 17344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:17:04.919507 17357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cce48f7885d42d1bbc2479f80f5b9eb. Latest consensus state: current_term: 1 leader_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } } }
I20260812 11:17:04.919647 17357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:04.919919 17355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cce48f7885d42d1bbc2479f80f5b9eb" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37453 } } }
I20260812 11:17:04.920019 17355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:04.920106 17373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:17:04.923017 17373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:17:04.928136 17373 catalog_manager.cc:1347] Generated new cluster ID: fceac0293f1b438e875c53fe9e888d4a
I20260812 11:17:04.928200 17373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:17:04.944149 17373 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:17:04.944921 17373 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:17:04.956223 17373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cce48f7885d42d1bbc2479f80f5b9eb: Generated new TSK 0
I20260812 11:17:04.956799 17373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:17:04.968291 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--builtin_ntp_servers=127.30.255.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 11:17:05.077171 17388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:05.077395 17388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:05.077442 17388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:05.081358 17388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:05.081403 17388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:05.081476 17388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:17:05.086069 17388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:05.086968 17388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:05.088218 17388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:05.097087 17397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.097170 17398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.097201 17388 server_base.cc:1034] running on GCE node
W20260812 11:17:05.097095 17401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.097689 17388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:05.098248 17388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:05.099395 17388 hybrid_clock.cc:648] HybridClock initialized: now 1786533425099369 us; error 43 us; skew 500 ppm
I20260812 11:17:05.101049 17388 webserver.cc:460] Webserver started at http://127.30.255.129:38915/ using document root <none> and password file <none>
I20260812 11:17:05.101567 17388 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:05.101631 17388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:05.101807 17388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:05.103752 17388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "4ff2930812df4b3ba570d7a13bf03b55"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "6886ead672a23a1266210f264a26a82e"
server_key_iv: "840ed86203e26a02b06ae1864b28f2b2"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.104272 17388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "4ff2930812df4b3ba570d7a13bf03b55"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "6886ead672a23a1266210f264a26a82e"
server_key_iv: "840ed86203e26a02b06ae1864b28f2b2"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.107607 17388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:17:05.109822 17412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.110918 17388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:17:05.111022 17388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "4ff2930812df4b3ba570d7a13bf03b55"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "6886ead672a23a1266210f264a26a82e"
server_key_iv: "840ed86203e26a02b06ae1864b28f2b2"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.111114 17388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:05.132779 17388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:05.133359 17388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:05.133513 17388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:05.134017 17388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:05.134853 17388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:05.134900 17388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.134951 17388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:05.134980 17388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.142851 17388 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:43789
I20260812 11:17:05.142904 17632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:43789 every 8 connection(s)
I20260812 11:17:05.143658 17388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:17:05.144932 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 17388
I20260812 11:17:05.145035 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:17:05.145359 31742 external_mini_cluster.cc:1428] Setting key 42acc0fc588810384c0b250c600c8204
I20260812 11:17:05.147219 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--builtin_ntp_servers=127.30.255.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 11:17:05.152252 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.145340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:42297 (local address 127.30.255.190:37453)
0812 11:17:05.145488 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.145493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.146505 (+ 1012us) server_negotiation.cc:407] Connection header received
0812 11:17:05.147920 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:05.147924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:05.147977 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:05.148058 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:05.148857 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.149389 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.150496 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.150721 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.150809 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.151800 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:05.151804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:05.151807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:05.151820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:05.151839 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:05.151847 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:05.151906 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:05.152049 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 11:17:05.152109 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:17:05.153061 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.145699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37453 (local address 127.30.255.129:42297)
0812 11:17:05.146378 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:05.146426 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:05.147709 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:05.148195 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:05.148201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:05.148279 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:05.148721 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.148733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.149513 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.149516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.150377 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.150386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.150859 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.150863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.151008 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.151559 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:05.151560 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:05.151597 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:05.151656 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:05.151916 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:05.151926 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:05.151932 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:17:05.152031 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 11:17:05.152308 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":104,"threads_started":1}
I20260812 11:17:05.154215 17634 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37453
I20260812 11:17:05.154424 17634 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:05.154891 17634 heartbeater.cc:511] Master 127.30.255.190:37453 requested a full tablet report, sending...
I20260812 11:17:05.156504 17288 ts_manager.cc:194] Registered new tserver with Master: 4ff2930812df4b3ba570d7a13bf03b55 (127.30.255.129:43789)
I20260812 11:17:05.158051 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.129:42297
W20260812 11:17:05.256443 17642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:05.256666 17642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:05.256699 17642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:05.260550 17642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:05.260592 17642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:05.260645 17642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:17:05.265141 17642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:05.265959 17642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:05.267115 17642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:17:05.275837 17642 server_base.cc:1034] running on GCE node
W20260812 11:17:05.275792 17651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.275851 17654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.275794 17652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.276382 17642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:05.276947 17642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:05.278105 17642 hybrid_clock.cc:648] HybridClock initialized: now 1786533425278092 us; error 27 us; skew 500 ppm
I20260812 11:17:05.279729 17642 webserver.cc:460] Webserver started at http://127.30.255.130:44327/ using document root <none> and password file <none>
I20260812 11:17:05.280236 17642 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:05.280300 17642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:05.280493 17642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:05.282343 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "b48b4fc734fc4e19b0e564c5fe6c39ef"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2e181ab8b26ee782cd19cbef5a917cd2"
server_key_iv: "3fd6836739fbebd879447261c0578829"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.282860 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "b48b4fc734fc4e19b0e564c5fe6c39ef"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2e181ab8b26ee782cd19cbef5a917cd2"
server_key_iv: "3fd6836739fbebd879447261c0578829"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.286255 17642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:17:05.288496 17675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.289536 17642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:17:05.289645 17642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "b48b4fc734fc4e19b0e564c5fe6c39ef"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2e181ab8b26ee782cd19cbef5a917cd2"
server_key_iv: "3fd6836739fbebd879447261c0578829"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.289729 17642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:05.311156 17642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:05.311746 17642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:05.311882 17642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:05.312340 17642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:05.313149 17642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:05.313196 17642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.313234 17642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:05.313262 17642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.321520 17642 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:35707
I20260812 11:17:05.321556 17888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:35707 every 8 connection(s)
I20260812 11:17:05.322335 17642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:17:05.323573 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 17642
I20260812 11:17:05.323668 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:17:05.323976 31742 external_mini_cluster.cc:1428] Setting key 043230929844cda8e733e1c570bb56f8
I20260812 11:17:05.326292 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--builtin_ntp_servers=127.30.255.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 11:17:05.331992 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.324044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37637 (local address 127.30.255.190:37453)
0812 11:17:05.324182 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.324187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.325264 (+ 1077us) server_negotiation.cc:407] Connection header received
0812 11:17:05.326880 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:05.326883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:05.326930 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:05.326995 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:05.328112 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.328909 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.330049 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.330360 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.330463 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.331511 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:05.331516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:05.331519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:05.331534 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:05.331570 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:05.331582 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:05.331664 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:05.331774 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 11:17:05.331842 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:17:05.332783 17892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.324384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37453 (local address 127.30.255.130:37637)
0812 11:17:05.325104 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:05.325156 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:05.326667 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:05.327169 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:05.327178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:05.327307 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:05.327947 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.327965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.329032 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.329035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.329917 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.329926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.330485 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.330488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.330635 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.331219 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:05.331220 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:05.331258 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:05.331354 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:05.331660 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:05.331664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:05.331668 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:17:05.331751 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 11:17:05.331961 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":141,"threads_started":1}
I20260812 11:17:05.333825 17889 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37453
I20260812 11:17:05.334065 17889 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:05.334702 17889 heartbeater.cc:511] Master 127.30.255.190:37453 requested a full tablet report, sending...
I20260812 11:17:05.335858 17288 ts_manager.cc:194] Registered new tserver with Master: b48b4fc734fc4e19b0e564c5fe6c39ef (127.30.255.130:35707)
I20260812 11:17:05.336416 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.130:37637
W20260812 11:17:05.435954 17894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:05.436167 17894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:05.436201 17894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:05.440084 17894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:05.440128 17894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:05.440183 17894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:17:05.444696 17894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:05.445546 17894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:05.446753 17894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:05.455353 17905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.455399 17906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.455363 17908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.455605 17894 server_base.cc:1034] running on GCE node
I20260812 11:17:05.456020 17894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:05.456635 17894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:05.457834 17894 hybrid_clock.cc:648] HybridClock initialized: now 1786533425457817 us; error 28 us; skew 500 ppm
I20260812 11:17:05.459499 17894 webserver.cc:460] Webserver started at http://127.30.255.131:46203/ using document root <none> and password file <none>
I20260812 11:17:05.460037 17894 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:05.460106 17894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:05.460320 17894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:05.462260 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "23d3330008144907a88506187c720cf0"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2ea997a6951b168a54bc68b29f30ffa4"
server_key_iv: "fd794bec2f3589b76c74c7bb35c95dbb"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.462795 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "23d3330008144907a88506187c720cf0"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2ea997a6951b168a54bc68b29f30ffa4"
server_key_iv: "fd794bec2f3589b76c74c7bb35c95dbb"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.466302 17894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:17:05.468641 17919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.469749 17894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:17:05.469856 17894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "23d3330008144907a88506187c720cf0"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "2ea997a6951b168a54bc68b29f30ffa4"
server_key_iv: "fd794bec2f3589b76c74c7bb35c95dbb"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.469945 17894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:05.499511 17894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:05.500165 17894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:05.500327 17894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:05.500844 17894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:05.501889 17894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:05.501948 17894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.501999 17894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:05.502029 17894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.509959 17894 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:34969
I20260812 11:17:05.509984 18144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:34969 every 8 connection(s)
I20260812 11:17:05.510732 17894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:17:05.513095 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 17894
I20260812 11:17:05.513226 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:17:05.513509 31742 external_mini_cluster.cc:1428] Setting key 0483bd8cbf313ca07e964298b51ad58e
I20260812 11:17:05.519028 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.512303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35795 (local address 127.30.255.190:37453)
0812 11:17:05.512416 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.512419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.513403 (+ 984us) server_negotiation.cc:407] Connection header received
0812 11:17:05.514859 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:05.514862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:05.514921 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:05.515024 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:05.515991 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.516507 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.517525 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.517739 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.517810 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.518678 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:05.518681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:05.518683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:05.518693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:05.518710 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:05.518717 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:05.518769 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:05.518870 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 11:17:05.518921 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:17:05.519843 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.512628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37453 (local address 127.30.255.131:35795)
0812 11:17:05.513256 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:05.513309 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:05.514657 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:05.515292 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:05.515302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:05.515388 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:05.515867 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.515881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.516596 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.516599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.517436 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:05.517445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:05.517842 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:05.517845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:05.517973 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:05.518460 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:05.518462 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:05.518498 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:05.518567 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:05.518776 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:05.518781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:05.518784 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 11:17:05.518866 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 11:17:05.519065 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
I20260812 11:17:05.520840 18145 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37453
I20260812 11:17:05.521034 18145 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:05.521474 18145 heartbeater.cc:511] Master 127.30.255.190:37453 requested a full tablet report, sending...
I20260812 11:17:05.522368 17288 ts_manager.cc:194] Registered new tserver with Master: 23d3330008144907a88506187c720cf0 (127.30.255.131:34969)
I20260812 11:17:05.522897 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.131:35795
I20260812 11:17:05.527019 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:17:05.529510 17350 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:05.528786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.30.255.190:37453)
0812 11:17:05.528889 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.528892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.529034 (+ 142us) server_negotiation.cc:407] Connection header received
0812 11:17:05.529147 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:05.529149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:05.529185 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:05.529240 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:05.529419 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51700: BlockingRecv error: recv got EOF from 127.0.0.1:51700 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:17:05.529891 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 17388
I20260812 11:17:05.570546 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 17642
I20260812 11:17:05.582651 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 17894
I20260812 11:17:05.597724 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 17237
2026-08-12T11:17:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (929 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T11:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:17:05Z Disabled control of system clock
I20260812 11:17:05.629660 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44069
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:17:05.734270 18173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:05.734489 18173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:05.734540 18173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:05.738399 18173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:17:05.738448 18173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:05.738471 18173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:05.738490 18173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:17:05.738510 18173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:17:05.743036 18173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:44069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:05.743943 18173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:05.745149 18173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:17:05.752952 18173 server_base.cc:1034] running on GCE node
W20260812 11:17:05.752910 18179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.752908 18181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.752928 18178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.753686 18173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:05.754660 18173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:05.755805 18173 hybrid_clock.cc:648] HybridClock initialized: now 1786533425755781 us; error 34 us; skew 500 ppm
I20260812 11:17:05.757426 18173 webserver.cc:460] Webserver started at http://127.30.255.190:37057/ using document root <none> and password file <none>
I20260812 11:17:05.757930 18173 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:05.757994 18173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:05.758181 18173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:05.760070 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "736b95d3e5a246e796332e35a82ed331"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "43df912f2f2af9688ec49549de880e44"
server_key_iv: "549aa8c99dd3c1554430f27116b9ae66"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.760581 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "736b95d3e5a246e796332e35a82ed331"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "43df912f2f2af9688ec49549de880e44"
server_key_iv: "549aa8c99dd3c1554430f27116b9ae66"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.763904 18173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:17:05.766072 18192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.767145 18173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:17:05.767252 18173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "736b95d3e5a246e796332e35a82ed331"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "43df912f2f2af9688ec49549de880e44"
server_key_iv: "549aa8c99dd3c1554430f27116b9ae66"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.767366 18173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:05.771512 18173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:05.772042 18173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:05.772204 18173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:05.778317 18173 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:44069
I20260812 11:17:05.778330 18293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:44069 every 8 connection(s)
I20260812 11:17:05.779109 18173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:17:05.782027 18295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:05.784273 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 18173
I20260812 11:17:05.784396 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:17:05.784700 31742 external_mini_cluster.cc:1428] Setting key 69f5bb050500d342a4eebf63f4a2246e
I20260812 11:17:05.787732 18295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331: Bootstrap starting.
I20260812 11:17:05.789770 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.786039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.30.255.190:44069)
0812 11:17:05.786631 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.786644 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.786675 (+ 31us) server_negotiation.cc:407] Connection header received
0812 11:17:05.787311 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:05.787335 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:05.787665 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:05.787960 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:05.788655 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:05.788660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:05.788672 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:05.788684 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:05.788739 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:05.788753 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:05.788876 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:05.788947 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 11:17:05.789175 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":129,"threads_started":1}
I20260812 11:17:05.789866 18295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:05.790576 18295 log.cc:824] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:05.792440 18295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331: No bootstrap required, opened a new log
I20260812 11:17:05.795591 18295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } }
I20260812 11:17:05.795787 18295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:05.795826 18295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736b95d3e5a246e796332e35a82ed331, State: Initialized, Role: FOLLOWER
I20260812 11:17:05.796181 18295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } }
I20260812 11:17:05.796296 18295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:17:05.796339 18295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:17:05.796406 18295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:05.797164 18295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } }
I20260812 11:17:05.797488 18295 leader_election.cc:302] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 736b95d3e5a246e796332e35a82ed331; no voters:
I20260812 11:17:05.797808 18295 leader_election.cc:288] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:17:05.797891 18302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:05.798089 18302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [term 1 LEADER]: Becoming Leader. State: Replica: 736b95d3e5a246e796332e35a82ed331, State: Running, Role: LEADER
I20260812 11:17:05.798398 18302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } }
I20260812 11:17:05.798890 18295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:17:05.799835 18306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 736b95d3e5a246e796332e35a82ed331. Latest consensus state: current_term: 1 leader_uuid: "736b95d3e5a246e796332e35a82ed331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } } }
I20260812 11:17:05.799966 18306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:05.800215 18305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "736b95d3e5a246e796332e35a82ed331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b95d3e5a246e796332e35a82ed331" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44069 } } }
I20260812 11:17:05.800304 18305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331 [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:05.800323 18328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:17:05.803154 18328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:17:05.808135 18328 catalog_manager.cc:1347] Generated new cluster ID: e4ec2c97fd624215b792b777502fffd8
I20260812 11:17:05.808194 18328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:17:05.820179 18328 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:17:05.820425 18328 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:17:05.826865 18328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 736b95d3e5a246e796332e35a82ed331: Generated new TSK 0
I20260812 11:17:05.827549 18328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:17:05.838085 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--builtin_ntp_servers=127.30.255.148:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:17:05.945922 18342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:05.946151 18342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:05.946201 18342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:05.950084 18342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:05.950130 18342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:05.950201 18342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:17:05.954710 18342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:05.955614 18342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:05.956830 18342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:05.966148 18348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.966195 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:05.966171 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:05.966388 18342 server_base.cc:1034] running on GCE node
I20260812 11:17:05.966953 18342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:05.967619 18342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:05.968775 18342 hybrid_clock.cc:648] HybridClock initialized: now 1786533425968756 us; error 28 us; skew 500 ppm
I20260812 11:17:05.970422 18342 webserver.cc:460] Webserver started at http://127.30.255.129:39299/ using document root <none> and password file <none>
I20260812 11:17:05.970997 18342 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:05.971062 18342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:05.971287 18342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:05.973289 18342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "b721f1531a644457a4a7a86706c88635"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "d0ad6f78367ed0f00bac8c619c965f5d"
server_key_iv: "97ffdf2019a014ee67d2754e6076ea54"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.973807 18342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "b721f1531a644457a4a7a86706c88635"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "d0ad6f78367ed0f00bac8c619c965f5d"
server_key_iv: "97ffdf2019a014ee67d2754e6076ea54"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.977852 18342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:17:05.980316 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.981508 18342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:17:05.981613 18342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "b721f1531a644457a4a7a86706c88635"
format_stamp: "Formatted at 2026-08-12 11:17:05 on dist-test-slave-gjrz"
server_key: "d0ad6f78367ed0f00bac8c619c965f5d"
server_key_iv: "97ffdf2019a014ee67d2754e6076ea54"
server_key_version: "encryptionkey@0"
I20260812 11:17:05.981730 18342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:05.986006 18342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:05.986564 18342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:05.986713 18342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:05.987216 18342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:05.988070 18342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:05.988117 18342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.988152 18342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:05.988175 18342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:05.995988 18342 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:43775
I20260812 11:17:05.996022 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:43775 every 8 connection(s)
I20260812 11:17:05.996747 18342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:17:06.002358 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.998321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:54597 (local address 127.30.255.190:44069)
0812 11:17:05.998449 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:05.998452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:05.999508 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 11:17:06.000786 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:06.000790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:06.000836 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:06.000909 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:06.001927 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:06.001930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:06.001933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:06.001946 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:06.001965 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:06.001974 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:06.002049 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:06.002182 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 11:17:06.002227 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:17:06.003087 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:05.998656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44069 (local address 127.30.255.129:54597)
0812 11:17:05.999363 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:05.999403 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:06.000584 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:06.001076 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:06.001083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:06.001176 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:06.001669 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:06.001671 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:06.001707 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:06.001773 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:06.002043 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:06.002052 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:06.002058 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:17:06.002214 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 11:17:06.002438 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":108,"threads_started":1}
I20260812 11:17:06.004197 18593 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44069
I20260812 11:17:06.004393 18593 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:06.004572 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 18342
I20260812 11:17:06.004688 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:17:06.004824 18593 heartbeater.cc:511] Master 127.30.255.190:44069 requested a full tablet report, sending...
I20260812 11:17:06.005007 31742 external_mini_cluster.cc:1428] Setting key fa8745521c54fada2186a64bb6bc7577
I20260812 11:17:06.006404 18222 ts_manager.cc:194] Registered new tserver with Master: b721f1531a644457a4a7a86706c88635 (127.30.255.129:43775)
I20260812 11:17:06.006955 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--builtin_ntp_servers=127.30.255.148:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:17:06.113626 18600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:06.113840 18600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:06.113883 18600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:06.117722 18600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:06.117766 18600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:06.117816 18600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:17:06.122267 18600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:06.123090 18600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:06.124285 18600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:06.133041 18617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:06.133033 18623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:06.133019 18619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:06.133133 18600 server_base.cc:1034] running on GCE node
I20260812 11:17:06.133581 18600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:06.134109 18600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:06.135289 18600 hybrid_clock.cc:648] HybridClock initialized: now 1786533426135234 us; error 49 us; skew 500 ppm
I20260812 11:17:06.136909 18600 webserver.cc:460] Webserver started at http://127.30.255.130:33749/ using document root <none> and password file <none>
I20260812 11:17:06.137422 18600 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:06.137486 18600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:06.137667 18600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:06.139602 18600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "31dc810b529849bc970ef060d812b49f"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "6f23e07402775f7d2c678a9ac2fa2200"
server_key_iv: "9e14cd41b808e00f010acb563f0828d4"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.140141 18600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "31dc810b529849bc970ef060d812b49f"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "6f23e07402775f7d2c678a9ac2fa2200"
server_key_iv: "9e14cd41b808e00f010acb563f0828d4"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.143608 18600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:17:06.145869 18632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.146989 18600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:17:06.147104 18600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "31dc810b529849bc970ef060d812b49f"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "6f23e07402775f7d2c678a9ac2fa2200"
server_key_iv: "9e14cd41b808e00f010acb563f0828d4"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.147197 18600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:06.151379 18600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:06.151911 18600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:06.152060 18600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:06.152546 18600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:06.153342 18600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:06.153388 18600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.153445 18600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:06.153472 18600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.161247 18600 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:36441
I20260812 11:17:06.161276 18842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:36441 every 8 connection(s)
I20260812 11:17:06.162021 18600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:17:06.162948 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 18600
I20260812 11:17:06.163074 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:17:06.163408 31742 external_mini_cluster.cc:1428] Setting key 4509ca5e285d7557064da0b0e8d0082a
I20260812 11:17:06.165479 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--builtin_ntp_servers=127.30.255.148:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 11:17:06.167987 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:06.163841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37397 (local address 127.30.255.190:44069)
0812 11:17:06.163991 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:06.163995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:06.165074 (+ 1079us) server_negotiation.cc:407] Connection header received
0812 11:17:06.166348 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:06.166352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:06.166394 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:06.166472 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:06.167591 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:06.167594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:06.167596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:06.167608 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:06.167630 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:06.167637 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:06.167702 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:06.167840 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:17:06.167873 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 11:17:06.168915 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:06.164221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44069 (local address 127.30.255.130:37397)
0812 11:17:06.164936 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:06.164983 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:06.166152 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:06.166640 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:06.166647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:06.166728 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:06.167287 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:06.167290 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:06.167342 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:06.167428 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:06.167708 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:06.167716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:06.167721 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 11:17:06.167881 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 11:17:06.168222 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260812 11:17:06.169955 18843 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44069
I20260812 11:17:06.170183 18843 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:06.170539 18843 heartbeater.cc:511] Master 127.30.255.190:44069 requested a full tablet report, sending...
I20260812 11:17:06.171635 18222 ts_manager.cc:194] Registered new tserver with Master: 31dc810b529849bc970ef060d812b49f (127.30.255.130:36441)
W20260812 11:17:06.273710 18847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:06.273923 18847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:06.273958 18847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:06.277746 18847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:06.277787 18847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:06.277838 18847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:17:06.282265 18847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:06.283118 18847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:06.284317 18847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:06.293295 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:06.293294 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:06.293309 18859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:06.293529 18847 server_base.cc:1034] running on GCE node
I20260812 11:17:06.293871 18847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:06.294399 18847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:06.295562 18847 hybrid_clock.cc:648] HybridClock initialized: now 1786533426295544 us; error 29 us; skew 500 ppm
I20260812 11:17:06.297147 18847 webserver.cc:460] Webserver started at http://127.30.255.131:43881/ using document root <none> and password file <none>
I20260812 11:17:06.297653 18847 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:06.297716 18847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:06.297886 18847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:06.299896 18847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "7526e12cd723478784fe92d83852c978"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "423d77f8c9c10b42102d783bb3eaff74"
server_key_iv: "8468ca2cd0f40a831bbfdbf097d84d09"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.300421 18847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "7526e12cd723478784fe92d83852c978"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "423d77f8c9c10b42102d783bb3eaff74"
server_key_iv: "8468ca2cd0f40a831bbfdbf097d84d09"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.303901 18847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:17:06.306383 18869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.307519 18847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:17:06.307618 18847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "7526e12cd723478784fe92d83852c978"
format_stamp: "Formatted at 2026-08-12 11:17:06 on dist-test-slave-gjrz"
server_key: "423d77f8c9c10b42102d783bb3eaff74"
server_key_iv: "8468ca2cd0f40a831bbfdbf097d84d09"
server_key_version: "encryptionkey@0"
I20260812 11:17:06.307708 18847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:06.311885 18847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:06.312403 18847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:06.312557 18847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:06.313035 18847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:06.313832 18847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:06.313885 18847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.313933 18847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:06.313961 18847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:06.321713 18847 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:35433
I20260812 11:17:06.321769 19082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:35433 every 8 connection(s)
I20260812 11:17:06.322477 18847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:17:06.323004 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 18847
I20260812 11:17:06.323130 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:17:06.323462 31742 external_mini_cluster.cc:1428] Setting key 68175dd2e3eb21683a07521199c0d55e
I20260812 11:17:06.328193 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:06.324155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:43415 (local address 127.30.255.190:44069)
0812 11:17:06.324271 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:06.324275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:06.325401 (+ 1126us) server_negotiation.cc:407] Connection header received
0812 11:17:06.326695 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:06.326699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:06.326742 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:06.326813 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:06.327787 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:06.327790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:06.327793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:17:06.327805 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:06.327826 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:17:06.327833 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:06.327904 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:06.328029 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 11:17:06.328070 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:17:06.328946 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:06.324615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44069 (local address 127.30.255.131:43415)
0812 11:17:06.325282 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:06.325320 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:06.326485 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:06.326975 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:06.326982 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:06.327071 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:06.327549 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:17:06.327551 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:06.327589 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:17:06.327653 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:06.327902 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:06.327910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:06.327916 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:17:06.328070 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 11:17:06.328293 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260812 11:17:06.329886 19084 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44069
I20260812 11:17:06.330060 19084 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:06.330381 19084 heartbeater.cc:511] Master 127.30.255.190:44069 requested a full tablet report, sending...
I20260812 11:17:06.331329 18222 ts_manager.cc:194] Registered new tserver with Master: 7526e12cd723478784fe92d83852c978 (127.30.255.131:35433)
I20260812 11:17:06.336941 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:17:06.339521 18299 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:17:06.338675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.30.255.190:44069)
0812 11:17:06.338792 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:06.338796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:06.338962 (+ 166us) server_negotiation.cc:407] Connection header received
0812 11:17:06.339076 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:06.339079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:06.339110 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:06.339168 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:06.339416 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41790: BlockingRecv error: recv got EOF from 127.0.0.1:41790 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:17:06.339673 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 18342
I20260812 11:17:06.357596 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 18600
I20260812 11:17:06.367910 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 18847
I20260812 11:17:06.387912 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 18173
2026-08-12T11:17:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (792 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:17:06 dist-test-slave-gjrz krb5kdc[19103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:17:06 dist-test-slave-gjrz krb5kdc[19103](info): set up 2 sockets
Aug 12 11:17:06 dist-test-slave-gjrz krb5kdc[19103](info): commencing operation
krb5kdc: starting...
W20260812 11:17:07.577320 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:17:07 dist-test-slave-gjrz krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:17:07.689945 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36179
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:07.795019 19127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:07.795240 19127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:07.795315 19127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:07.799158 19127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:17:07.799207 19127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:07.799226 19127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:07.799255 19127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:17:07.799302 19127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:17:07.804200 19127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:07.805018 19127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:07.805884 19127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:07.813591 19138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:07.813642 19134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:07.813642 19136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:07.813782 19127 server_base.cc:1034] running on GCE node
I20260812 11:17:07.814354 19127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:07.815342 19127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:07.816511 19127 hybrid_clock.cc:648] HybridClock initialized: now 1786533427816492 us; error 29 us; skew 500 ppm
Aug 12 11:17:07 dist-test-slave-gjrz krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:07.822924 19127 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:17:07.823747 19127 webserver.cc:460] Webserver started at http://127.30.255.190:42327/ using document root <none> and password file <none>
I20260812 11:17:07.824237 19127 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:07.824319 19127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:07.824514 19127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:07.826396 19127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "0875e88f2c0f4108b7349ce67b673a0b"
format_stamp: "Formatted at 2026-08-12 11:17:07 on dist-test-slave-gjrz"
server_key: "6573baa212366ca73ee2f3d84f04f171"
server_key_iv: "8c67b73d2132de00554c74a2fa9a39c5"
server_key_version: "encryptionkey@0"
I20260812 11:17:07.826918 19127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "0875e88f2c0f4108b7349ce67b673a0b"
format_stamp: "Formatted at 2026-08-12 11:17:07 on dist-test-slave-gjrz"
server_key: "6573baa212366ca73ee2f3d84f04f171"
server_key_iv: "8c67b73d2132de00554c74a2fa9a39c5"
server_key_version: "encryptionkey@0"
I20260812 11:17:07.830295 19127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:17:07.832494 19149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:07.833573 19127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:17:07.833678 19127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0875e88f2c0f4108b7349ce67b673a0b"
format_stamp: "Formatted at 2026-08-12 11:17:07 on dist-test-slave-gjrz"
server_key: "6573baa212366ca73ee2f3d84f04f171"
server_key_iv: "8c67b73d2132de00554c74a2fa9a39c5"
server_key_version: "encryptionkey@0"
I20260812 11:17:07.833768 19127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:07.855757 19127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:07.858117 19127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:07.858266 19127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:07.864559 19127 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:36179
I20260812 11:17:07.864800 19236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:36179 every 8 connection(s)
I20260812 11:17:07.865447 19127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:17:07.867975 19237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:07.872987 19237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: Bootstrap starting.
I20260812 11:17:07.875065 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 19127
I20260812 11:17:07.875166 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:17:07.875231 19237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:07.875478 31742 external_mini_cluster.cc:1428] Setting key 4f599088381c468d14c8d9f2652edb5b
I20260812 11:17:07.876006 19237 log.cc:824] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:07.877794 19237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: No bootstrap required, opened a new log
I20260812 11:17:07.881695 19237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } }
I20260812 11:17:07.881964 19237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:07.882026 19237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0875e88f2c0f4108b7349ce67b673a0b, State: Initialized, Role: FOLLOWER
I20260812 11:17:07.882470 19237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } }
I20260812 11:17:07.882633 19237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:17:07.882701 19237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:17:07.882803 19237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 0 FOLLOWER]: Advancing to term 1
Aug 12 11:17:07 dist-test-slave-gjrz krb5kdc[19103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:17:07.883738 19237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } }
I20260812 11:17:07.884053 19237 leader_election.cc:302] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0875e88f2c0f4108b7349ce67b673a0b; no voters:
I20260812 11:17:07.884411 19237 leader_election.cc:288] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:17:07.884482 19247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:07.884976 19247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [term 1 LEADER]: Becoming Leader. State: Replica: 0875e88f2c0f4108b7349ce67b673a0b, State: Running, Role: LEADER
I20260812 11:17:07.885301 19247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } }
I20260812 11:17:07.885588 19237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:17:07.886994 19249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0875e88f2c0f4108b7349ce67b673a0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } } }
I20260812 11:17:07.887125 19249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:07.887531 19258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0875e88f2c0f4108b7349ce67b673a0b. Latest consensus state: current_term: 1 leader_uuid: "0875e88f2c0f4108b7349ce67b673a0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0875e88f2c0f4108b7349ce67b673a0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36179 } } }
I20260812 11:17:07.887694 19258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:07.887920 19280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:17:07.889521 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:07.876472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.30.255.190:36179)
0812 11:17:07.876883 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:07.876897 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:07.876930 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:17:07.877510 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:07.877525 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:07.877811 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:07.878098 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:07.878901 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:07.879877 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:07.881156 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:07.881603 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:07.881713 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:07.884413 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:07.884419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:07.884436 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:07.884467 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:07.886396 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:07.886885 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:07.886891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:07.886897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:07.887036 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:07.887392 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:07.887407 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:07.887408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:07.887889 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:07.888089 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:07.888524 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 11:17:07.888808 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260812 11:17:07.892066 19280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:17:07.897475 19280 catalog_manager.cc:1347] Generated new cluster ID: 6355538a420744618b666ba15255416c
I20260812 11:17:07.897536 19280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:17:07.906126 19280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:17:07.907240 19280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:17:07.916985 19280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: Generated new TSK 0
I20260812 11:17:07.917708 19280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:17:07.949600 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:08.057329 19293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:08.057543 19293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:08.057600 19293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:08.061437 19293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:08.061484 19293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:08.061558 19293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:17:08.066421 19293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:08.067286 19293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:08.068162 19293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:08.077250 19310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:08.077265 19311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:08.077260 19314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:08.077293 19293 server_base.cc:1034] running on GCE node
I20260812 11:17:08.077862 19293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:08.078423 19293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:08.079567 19293 hybrid_clock.cc:648] HybridClock initialized: now 1786533428079541 us; error 38 us; skew 500 ppm
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:08.082223 19293 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:17:08.082990 19293 webserver.cc:460] Webserver started at http://127.30.255.129:40193/ using document root <none> and password file <none>
I20260812 11:17:08.083516 19293 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:08.083585 19293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:08.083779 19293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:08.085597 19293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "703d44d9300a09514061baaa8d73b2c3"
server_key_iv: "57988fcfdfd597d91eb024fc690ec69b"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.086117 19293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "703d44d9300a09514061baaa8d73b2c3"
server_key_iv: "57988fcfdfd597d91eb024fc690ec69b"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.089676 19293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:17:08.091856 19322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.092934 19293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:17:08.093039 19293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "703d44d9300a09514061baaa8d73b2c3"
server_key_iv: "57988fcfdfd597d91eb024fc690ec69b"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.093130 19293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:08.112721 19293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:08.115186 19293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:08.115368 19293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:08.115890 19293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:08.116629 19293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:08.116674 19293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.116710 19293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:08.116735 19293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.124775 19293 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:38963
I20260812 11:17:08.124801 19558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:38963 every 8 connection(s)
I20260812 11:17:08.125542 19293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:17:08.125885 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 19293
I20260812 11:17:08.126010 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:17:08.126325 31742 external_mini_cluster.cc:1428] Setting key 5a176ef31a20237b6a4b9080a75998e9
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:17:08.139662 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.127369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:38223 (local address 127.30.255.190:36179)
0812 11:17:08.127508 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.127512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.128394 (+ 882us) server_negotiation.cc:407] Connection header received
0812 11:17:08.129727 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.129730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.129781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.129847 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:08.131297 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.131819 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.133457 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.133817 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.133926 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.136308 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:08.136313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:08.136316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:08.136346 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:08.137801 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.138303 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.138305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.138307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.138378 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.138637 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.138639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.138640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.138797 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:08.138867 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.139388 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.139496 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:17:08.140455 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.127783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36179 (local address 127.30.255.129:38223)
0812 11:17:08.128261 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.128301 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.129478 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.130044 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.130053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.130507 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:08.131132 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.131152 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.131948 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.131952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.133335 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.133346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.133943 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.133945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.134081 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.134574 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:08.134589 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:08.136138 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:08.137913 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.137919 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.137933 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.138224 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.138441 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.138444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.138445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.138573 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.138911 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:08.138917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:08.139176 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.139382 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.139681 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":133,"threads_started":1}
I20260812 11:17:08.141808 19560 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36179
I20260812 11:17:08.142071 19560 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:08.142596 19560 heartbeater.cc:511] Master 127.30.255.190:36179 requested a full tablet report, sending...
I20260812 11:17:08.144052 19179 ts_manager.cc:194] Registered new tserver with Master: fa348dbf30dd4d44b28a51de5a3ddeb7 (127.30.255.129:38963)
I20260812 11:17:08.145115 19179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:38223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:17:08.154129 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:08.259769 19568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:08.260000 19568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:08.260056 19568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:08.263960 19568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:08.264014 19568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:08.264091 19568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:17:08.268985 19568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:08.269836 19568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:08.270720 19568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:08.279716 19585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:08.279729 19588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:08.279709 19581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:08.279946 19568 server_base.cc:1034] running on GCE node
I20260812 11:17:08.280398 19568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:08.280988 19568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:08.282152 19568 hybrid_clock.cc:648] HybridClock initialized: now 1786533428282133 us; error 30 us; skew 500 ppm
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:08.284725 19568 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:17:08.285578 19568 webserver.cc:460] Webserver started at http://127.30.255.130:43023/ using document root <none> and password file <none>
I20260812 11:17:08.286154 19568 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:08.286229 19568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:08.286425 19568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:08.288343 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "d5e8f3b0e4cc41ff962133200686515b"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "26c076b993c189fc90d2faa34f47b06a"
server_key_iv: "ebec780c5351ba831230e9dc8e3c7e0e"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.288880 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "d5e8f3b0e4cc41ff962133200686515b"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "26c076b993c189fc90d2faa34f47b06a"
server_key_iv: "ebec780c5351ba831230e9dc8e3c7e0e"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.292322 19568 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 11:17:08.294462 19597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.295606 19568 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:17:08.295712 19568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "d5e8f3b0e4cc41ff962133200686515b"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "26c076b993c189fc90d2faa34f47b06a"
server_key_iv: "ebec780c5351ba831230e9dc8e3c7e0e"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.295801 19568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:08.312266 19568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:08.315016 19568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:08.315172 19568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:08.315719 19568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:08.316443 19568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:08.316488 19568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.316535 19568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:08.316563 19568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.324384 19568 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:43193
I20260812 11:17:08.324416 19828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:43193 every 8 connection(s)
I20260812 11:17:08.325146 19568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:17:08.329175 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 19568
I20260812 11:17:08.329274 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:17:08.329525 31742 external_mini_cluster.cc:1428] Setting key 0cea5c93b9eba3d6baf8d089656d9a40
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:17:08.338171 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.326699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:58397 (local address 127.30.255.190:36179)
0812 11:17:08.326835 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.326838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.327590 (+ 752us) server_negotiation.cc:407] Connection header received
0812 11:17:08.328871 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.328875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.328935 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.329023 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:08.330127 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.330702 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.331880 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.332187 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.332289 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.334601 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:08.334605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:08.334607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:08.334634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:08.336171 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.336643 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.336647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.336649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.336744 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.337084 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.337087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.337089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.337315 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:08.337416 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.337866 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.337995 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:17:08.338842 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.327009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36179 (local address 127.30.255.130:58397)
0812 11:17:08.327449 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.327488 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.328670 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.329168 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.329174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.329534 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:08.330000 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.330014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.330822 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.330826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.331748 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.331758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.332326 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.332329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.332466 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.332947 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:08.332963 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:08.334423 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:08.336280 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.336285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.336295 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.336537 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.336831 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.336834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.336836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.336970 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.337412 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:08.337417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:08.337631 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.337800 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.338073 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:17:08.339882 19829 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36179
I20260812 11:17:08.340075 19829 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:08.340463 19829 heartbeater.cc:511] Master 127.30.255.190:36179 requested a full tablet report, sending...
I20260812 11:17:08.341401 19179 ts_manager.cc:194] Registered new tserver with Master: d5e8f3b0e4cc41ff962133200686515b (127.30.255.130:43193)
I20260812 11:17:08.341990 19179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:58397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:17:08.356050 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:08.462162 19840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:08.462364 19840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:08.462406 19840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:08.466303 19840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:08.466351 19840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:08.466418 19840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:17:08.471380 19840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:08.472218 19840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:08.473096 19840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:08.481817 19854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:08.481868 19840 server_base.cc:1034] running on GCE node
W20260812 11:17:08.481817 19852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:08.481844 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:08.482385 19840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:08.482930 19840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:08.484082 19840 hybrid_clock.cc:648] HybridClock initialized: now 1786533428484071 us; error 26 us; skew 500 ppm
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:08.486625 19840 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:17:08.487499 19840 webserver.cc:460] Webserver started at http://127.30.255.131:34449/ using document root <none> and password file <none>
I20260812 11:17:08.487985 19840 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:08.488044 19840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:08.488243 19840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:17:08.490152 19840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "3d19d237cda549d982e93b80f2f7d6c9"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "6f13bbf25174d8ff496bf2d229732b4d"
server_key_iv: "66669397ac5a9d87a09165f4c0c6b6ee"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.490653 19840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "3d19d237cda549d982e93b80f2f7d6c9"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "6f13bbf25174d8ff496bf2d229732b4d"
server_key_iv: "66669397ac5a9d87a09165f4c0c6b6ee"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.494098 19840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:17:08.496341 19868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.497478 19840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:17:08.497596 19840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "3d19d237cda549d982e93b80f2f7d6c9"
format_stamp: "Formatted at 2026-08-12 11:17:08 on dist-test-slave-gjrz"
server_key: "6f13bbf25174d8ff496bf2d229732b4d"
server_key_iv: "66669397ac5a9d87a09165f4c0c6b6ee"
server_key_version: "encryptionkey@0"
I20260812 11:17:08.497685 19840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:08.523701 19840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:08.526185 19840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:08.526337 19840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:08.526834 19840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:08.527590 19840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:08.527634 19840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.527673 19840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:08.527701 19840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:08.536109 19840 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:41671
I20260812 11:17:08.536128 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:41671 every 8 connection(s)
I20260812 11:17:08.536890 19840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:17:08.541476 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 19840
I20260812 11:17:08.541581 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:17:08.541831 31742 external_mini_cluster.cc:1428] Setting key 453991d87b5ef2d56341d8f803590167
Aug 12 11:17:08 dist-test-slave-gjrz krb5kdc[19103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:17:08.549130 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.538532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35801 (local address 127.30.255.190:36179)
0812 11:17:08.538679 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.538684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.539498 (+ 814us) server_negotiation.cc:407] Connection header received
0812 11:17:08.540738 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.540740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.540786 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.540855 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:08.541974 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.542639 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.543753 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.543973 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.544059 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.546173 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:08.546177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:08.546180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:08.546205 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:08.547295 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.547744 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.547748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.547750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.547829 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.548128 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.548131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.548133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.548310 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:08.548394 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.548844 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.548960 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:17:08.549763 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.538904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36179 (local address 127.30.255.131:35801)
0812 11:17:08.539371 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.539411 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.540554 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.540996 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.541002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.541317 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:08.541811 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.541826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.542740 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.542743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.543629 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.543639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.544108 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.544112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.544264 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.544730 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:08.544746 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:08.546021 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:08.547393 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.547399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.547412 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.547653 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.547909 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:08.547912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:08.547914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:08.548034 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:08.548408 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:08.548415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:08.548629 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.548782 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.549062 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":118,"threads_started":1}
I20260812 11:17:08.550652 20066 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36179
I20260812 11:17:08.550827 20066 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:08.551205 20066 heartbeater.cc:511] Master 127.30.255.190:36179 requested a full tablet report, sending...
I20260812 11:17:08.552106 19179 ts_manager.cc:194] Registered new tserver with Master: 3d19d237cda549d982e93b80f2f7d6c9 (127.30.255.131:41671)
I20260812 11:17:08.552634 19179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:35801
I20260812 11:17:08.555171 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:17:08.562819 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.556902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.30.255.190:36179)
0812 11:17:08.557027 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.557031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.557176 (+ 145us) server_negotiation.cc:407] Connection header received
0812 11:17:08.557258 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.557260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.557294 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.557353 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:08.557896 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.558346 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.559054 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.559243 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.559343 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.560372 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:08.560374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:08.560376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:08.560396 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:08.561612 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.561941 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.561943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.561944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.562005 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:08.562187 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:08.562189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:08.562190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:08.562366 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:08.562459 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.562603 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.562682 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:17:08.565284 19179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:17:08.568107 19179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:17:08.581317 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.575348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:43193 (local address 127.0.0.1:35562)
0812 11:17:08.575843 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.575888 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.576016 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.576788 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.576793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.576817 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.576931 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.576937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578536 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.578539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.579809 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.579819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580895 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.581044 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.581065 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.581117 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.581169 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":104,"spinlock_wait_cycles":4352,"thread_start_us":62,"threads_started":1}
I20260812 11:17:08.581440 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.575510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.30.255.130:43193)
0812 11:17:08.576292 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.576303 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.576326 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:17:08.576405 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.576410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.576614 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.577098 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.577255 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578404 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.579939 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580749 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.581015 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.581146 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.581198 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.581287 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":156,"threads_started":1}
I20260812 11:17:08.581480 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.576311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.30.255.129:38963)
0812 11:17:08.576830 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.576842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.576864 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:17:08.576937 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.576942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.577122 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.577526 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.577691 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578731 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580130 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580665 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.580903 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.581248 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.581343 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":107,"threads_started":1}
I20260812 11:17:08.581526 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.575478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:41671 (local address 127.0.0.1:59520)
0812 11:17:08.575908 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.575920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.576016 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.577087 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.577090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.577108 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.577207 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.577213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578627 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.578631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.579944 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.579956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580915 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.581245 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.581269 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.581337 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.581401 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":214,"spinlock_wait_cycles":3584,"thread_start_us":71,"threads_started":1}
I20260812 11:17:08.581709 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.575986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38963 (local address 127.0.0.1:57648)
0812 11:17:08.576295 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.576307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.576391 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.577299 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.577301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.577317 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.577403 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.577407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578878 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.578881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.580009 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.580018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580777 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.580897 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.580949 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.581047 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.581167 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260812 11:17:08.582294 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.576300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.30.255.131:41671)
0812 11:17:08.576679 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.576689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.576709 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:17:08.576774 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.576779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.576944 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.577345 (+ 401us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.577466 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.578513 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.580071 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.580804 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.581036 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.581141 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.582068 (+ 927us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.582158 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
I20260812 11:17:08.584452 19692 tablet_service.cc:1467] Processing CreateTablet for tablet 09fb14c50bff4e6db22905c11d146f0f (DEFAULT_TABLE table=test-table [id=d43c86304e7a47429f791b7fdc0b7a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:08.584534 19429 tablet_service.cc:1467] Processing CreateTablet for tablet 09fb14c50bff4e6db22905c11d146f0f (DEFAULT_TABLE table=test-table [id=d43c86304e7a47429f791b7fdc0b7a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:08.584651 19957 tablet_service.cc:1467] Processing CreateTablet for tablet 09fb14c50bff4e6db22905c11d146f0f (DEFAULT_TABLE table=test-table [id=d43c86304e7a47429f791b7fdc0b7a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:08.585348 19692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fb14c50bff4e6db22905c11d146f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:08.585537 19429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fb14c50bff4e6db22905c11d146f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:08.585537 19957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fb14c50bff4e6db22905c11d146f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:08.591285 20099 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Bootstrap starting.
I20260812 11:17:08.592711 20101 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Bootstrap starting.
I20260812 11:17:08.592711 20100 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Bootstrap starting.
I20260812 11:17:08.594339 20099 tablet_bootstrap.cc:654] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:08.594789 20100 tablet_bootstrap.cc:654] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:08.595053 20099 log.cc:824] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:08.595206 20101 tablet_bootstrap.cc:654] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:08.596037 20101 log.cc:824] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:08.596037 20100 log.cc:824] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:08.596939 20099 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: No bootstrap required, opened a new log
I20260812 11:17:08.597173 20099 ts_tablet_manager.cc:1397] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:17:08.597829 20100 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: No bootstrap required, opened a new log
I20260812 11:17:08.598004 20100 ts_tablet_manager.cc:1397] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:17:08.598407 20101 tablet_bootstrap.cc:492] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: No bootstrap required, opened a new log
I20260812 11:17:08.598573 20101 ts_tablet_manager.cc:1397] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:17:08.600864 20100 raft_consensus.cc:348] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.601125 20100 raft_consensus.cc:374] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:08.601208 20100 raft_consensus.cc:729] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa348dbf30dd4d44b28a51de5a3ddeb7, State: Initialized, Role: FOLLOWER
I20260812 11:17:08.601151 20099 raft_consensus.cc:348] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.601393 20099 raft_consensus.cc:374] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:08.601455 20099 raft_consensus.cc:729] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e8f3b0e4cc41ff962133200686515b, State: Initialized, Role: FOLLOWER
I20260812 11:17:08.601665 20100 consensus_queue.cc:260] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.601862 20099 consensus_queue.cc:260] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.602522 19829 heartbeater.cc:503] Master 127.30.255.190:36179 was elected leader, sending a full tablet report...
I20260812 11:17:08.602662 20100 ts_tablet_manager.cc:1428] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:17:08.602545 20101 raft_consensus.cc:348] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.602759 20101 raft_consensus.cc:374] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:08.602799 20099 ts_tablet_manager.cc:1428] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:17:08.602808 20101 raft_consensus.cc:729] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d19d237cda549d982e93b80f2f7d6c9, State: Initialized, Role: FOLLOWER
I20260812 11:17:08.602945 19560 heartbeater.cc:503] Master 127.30.255.190:36179 was elected leader, sending a full tablet report...
I20260812 11:17:08.603150 20101 consensus_queue.cc:260] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.603689 20066 heartbeater.cc:503] Master 127.30.255.190:36179 was elected leader, sending a full tablet report...
I20260812 11:17:08.603962 20101 ts_tablet_manager.cc:1428] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 11:17:08.626683 19561 tablet.cc:2369] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:17:08.787798 20068 tablet.cc:2369] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:17:08.826201 19832 tablet.cc:2369] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:17:08.982529 20112 raft_consensus.cc:482] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:17:08.982728 20112 raft_consensus.cc:504] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.983873 20112 leader_election.cc:288] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5e8f3b0e4cc41ff962133200686515b (127.30.255.130:43193), 3d19d237cda549d982e93b80f2f7d6c9 (127.30.255.131:41671)
I20260812 11:17:08.988934 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.984258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:44021 (local address 127.30.255.130:43193)
0812 11:17:08.984386 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.984391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.984530 (+ 139us) server_negotiation.cc:407] Connection header received
0812 11:17:08.984678 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.984681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.984736 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.984820 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.985178 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.985977 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.987707 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.988310 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.988423 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.988475 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.988745 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.988807 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:17:08.988934 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.984261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:45679 (local address 127.30.255.131:41671)
0812 11:17:08.984429 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.984433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.984651 (+ 218us) server_negotiation.cc:407] Connection header received
0812 11:17:08.984750 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.984753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.984803 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.984869 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.985169 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.985977 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.987707 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.988310 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.988423 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.988475 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.988743 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.988811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 11:17:08.989005 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.984082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:43193 (local address 127.30.255.129:44021)
0812 11:17:08.984440 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.984455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.984576 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.984831 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.984834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.984852 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.985053 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.985067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.986079 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.986082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.987589 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.987599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.988431 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.988434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.988601 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.988650 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.988715 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.988823 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":231,"spinlock_wait_cycles":28928,"thread_start_us":102,"threads_started":1}
I20260812 11:17:08.989007 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.984113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:41671 (local address 127.30.255.129:45679)
0812 11:17:08.984573 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.984585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.984668 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.984880 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.984883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.984918 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.985059 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.985067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.986079 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.986082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.987587 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.987599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.988440 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.988443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.988605 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.988643 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.988720 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.988817 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":121,"spinlock_wait_cycles":3840,"thread_start_us":97,"threads_started":1}
I20260812 11:17:08.989681 20113 raft_consensus.cc:482] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:17:08.989694 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d19d237cda549d982e93b80f2f7d6c9" is_pre_election: true
I20260812 11:17:08.989832 19736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e8f3b0e4cc41ff962133200686515b" is_pre_election: true
I20260812 11:17:08.989837 20113 raft_consensus.cc:504] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.989998 19986 raft_consensus.cc:2454] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa348dbf30dd4d44b28a51de5a3ddeb7 in term 0.
I20260812 11:17:08.990262 19736 raft_consensus.cc:2454] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa348dbf30dd4d44b28a51de5a3ddeb7 in term 0.
I20260812 11:17:08.990475 19326 leader_election.cc:302] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d19d237cda549d982e93b80f2f7d6c9, fa348dbf30dd4d44b28a51de5a3ddeb7; no voters:
I20260812 11:17:08.990748 20112 raft_consensus.cc:2781] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:17:08.990908 20112 raft_consensus.cc:482] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:17:08.990968 20113 leader_election.cc:288] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d19d237cda549d982e93b80f2f7d6c9 (127.30.255.131:41671), fa348dbf30dd4d44b28a51de5a3ddeb7 (127.30.255.129:38963)
I20260812 11:17:08.991020 20112 raft_consensus.cc:3037] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:08.992183 20112 raft_consensus.cc:504] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.992880 19736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e8f3b0e4cc41ff962133200686515b"
I20260812 11:17:08.992980 19736 raft_consensus.cc:3037] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:08.993175 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d19d237cda549d982e93b80f2f7d6c9"
I20260812 11:17:08.993296 19986 raft_consensus.cc:3037] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:08.994192 19736 raft_consensus.cc:2454] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa348dbf30dd4d44b28a51de5a3ddeb7 in term 1.
I20260812 11:17:08.994488 19986 raft_consensus.cc:2454] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa348dbf30dd4d44b28a51de5a3ddeb7 in term 1.
I20260812 11:17:08.994494 19329 leader_election.cc:302] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5e8f3b0e4cc41ff962133200686515b, fa348dbf30dd4d44b28a51de5a3ddeb7; no voters:
I20260812 11:17:08.994728 20112 leader_election.cc:288] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [CANDIDATE]: Term 1 election: Requested vote from peers d5e8f3b0e4cc41ff962133200686515b (127.30.255.130:43193), 3d19d237cda549d982e93b80f2f7d6c9 (127.30.255.131:41671)
I20260812 11:17:08.994940 20126 raft_consensus.cc:2781] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:08.995044 20126 raft_consensus.cc:686] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [term 1 LEADER]: Becoming Leader. State: Replica: fa348dbf30dd4d44b28a51de5a3ddeb7, State: Running, Role: LEADER
I20260812 11:17:08.995424 20126 consensus_queue.cc:237] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } }
I20260812 11:17:08.995596 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.991447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38963 (local address 127.30.255.130:43567)
0812 11:17:08.991675 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.991688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.991776 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.992081 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.992084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.992106 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.992209 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.992216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.993125 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.993128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.994327 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.994337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.995191 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.995194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.995305 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.995327 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.995400 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.995460 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":45,"thread_start_us":68,"threads_started":1}
I20260812 11:17:08.995633 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.991751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:43567 (local address 127.30.255.129:38963)
0812 11:17:08.991881 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.991885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.991899 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:17:08.991935 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.991937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.991981 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.992071 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.992299 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.993004 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.994453 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.994961 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.995045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.995100 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.995467 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.995533 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:17:08.995756 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.991920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37877 (local address 127.30.255.131:41671)
0812 11:17:08.992025 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:08.992030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:08.992042 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:17:08.992079 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:08.992082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:08.992120 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:08.992194 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:08.992485 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.993150 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.994583 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.995110 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.995200 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.995240 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:08.995590 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 11:17:08.995657 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:17:08.996433 19463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "d5e8f3b0e4cc41ff962133200686515b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" is_pre_election: true
I20260812 11:17:08.996976 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:08.991110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:41671 (local address 127.30.255.130:37877)
0812 11:17:08.991333 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:08.991345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:08.991418 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:08.992233 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:08.992236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:08.992256 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:08.992364 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.992371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.993268 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.993271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.994442 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:08.994454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:08.995352 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:08.995355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:08.995447 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:08.995468 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:17:08.996165 (+ 697us) client_negotiation.cc:240] Negotiation successful
0812 11:17:08.996236 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260812 11:17:08.997349 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fb14c50bff4e6db22905c11d146f0f" candidate_uuid: "d5e8f3b0e4cc41ff962133200686515b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d19d237cda549d982e93b80f2f7d6c9" is_pre_election: true
I20260812 11:17:08.997486 19986 raft_consensus.cc:2379] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5e8f3b0e4cc41ff962133200686515b in current term 1: Already voted for candidate fa348dbf30dd4d44b28a51de5a3ddeb7 in this term.
I20260812 11:17:08.997874 19600 leader_election.cc:302] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5e8f3b0e4cc41ff962133200686515b; no voters: 3d19d237cda549d982e93b80f2f7d6c9, fa348dbf30dd4d44b28a51de5a3ddeb7
I20260812 11:17:08.998296 20113 raft_consensus.cc:2726] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:17:08.998819 19177 catalog_manager.cc:5495] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa348dbf30dd4d44b28a51de5a3ddeb7 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa348dbf30dd4d44b28a51de5a3ddeb7" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38963 } health_report { overall_health: HEALTHY } } }
I20260812 11:17:09.027374 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:09.024110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.30.255.129:38963)
0812 11:17:09.024216 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:09.024219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:09.024232 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:17:09.024375 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:09.024377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:09.024426 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:09.024527 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:17:09.024775 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:09.025254 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:09.026067 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:09.026271 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:09.026355 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:09.026638 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:17:09.027004 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:17:09.027108 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:09.027186 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 11:17:09.027231 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:17:09.032403 19986 raft_consensus.cc:1261] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 1 FOLLOWER]: Refusing update from remote peer fa348dbf30dd4d44b28a51de5a3ddeb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:17:09.032441 19736 raft_consensus.cc:1261] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 1 FOLLOWER]: Refusing update from remote peer fa348dbf30dd4d44b28a51de5a3ddeb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:17:09.033030 20112 consensus_queue.cc:1035] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d19d237cda549d982e93b80f2f7d6c9" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:17:09.033195 20126 consensus_queue.cc:1035] T 09fb14c50bff4e6db22905c11d146f0f P fa348dbf30dd4d44b28a51de5a3ddeb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5e8f3b0e4cc41ff962133200686515b" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 43193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:17:09.040186 20134 mvcc.cc:205] Tried to move back new op lower bound from 7317640925310529536 to 7317640925168427008. Current Snapshot: MvccSnapshot[applied={T|T < 7317640925310529536}]
I20260812 11:17:09.050204 19177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:17:09.050374 19177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:17:09.052672 19177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0875e88f2c0f4108b7349ce67b673a0b: Sending DeleteTablet for 3 replicas of tablet 09fb14c50bff4e6db22905c11d146f0f
I20260812 11:17:09.053334 19692 tablet_service.cc:1514] Processing DeleteTablet for tablet 09fb14c50bff4e6db22905c11d146f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:09 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:35562
I20260812 11:17:09.053589 19957 tablet_service.cc:1514] Processing DeleteTablet for tablet 09fb14c50bff4e6db22905c11d146f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:09 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:59520
I20260812 11:17:09.053812 20148 tablet_replica.cc:330] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: stopping tablet replica
I20260812 11:17:09.053974 20149 tablet_replica.cc:330] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: stopping tablet replica
I20260812 11:17:09.054015 19429 tablet_service.cc:1514] Processing DeleteTablet for tablet 09fb14c50bff4e6db22905c11d146f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:09 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:57648
I20260812 11:17:09.054117 20148 raft_consensus.cc:2229] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:17:09.054240 20149 raft_consensus.cc:2229] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:17:09.054385 20148 raft_consensus.cc:2258] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:17:09.054450 20149 raft_consensus.cc:2258] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:17:09.054577 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 19293
I20260812 11:17:09.061141 20148 ts_tablet_manager.cc:1905] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:17:09.063985 20148 ts_tablet_manager.cc:1918] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:17:09.064087 20148 log.cc:1196] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/09fb14c50bff4e6db22905c11d146f0f
I20260812 11:17:09.064808 20148 ts_tablet_manager.cc:1939] T 09fb14c50bff4e6db22905c11d146f0f P d5e8f3b0e4cc41ff962133200686515b: Deleting consensus metadata
I20260812 11:17:09.066164 19155 catalog_manager.cc:4841] TS d5e8f3b0e4cc41ff962133200686515b (127.30.255.130:43193): tablet 09fb14c50bff4e6db22905c11d146f0f (table test-table [id=d43c86304e7a47429f791b7fdc0b7a29]) successfully deleted
I20260812 11:17:09.068145 20149 ts_tablet_manager.cc:1905] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:17:09.070096 20149 ts_tablet_manager.cc:1918] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:17:09.070180 20149 log.cc:1196] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/09fb14c50bff4e6db22905c11d146f0f
I20260812 11:17:09.070415 20149 ts_tablet_manager.cc:1939] T 09fb14c50bff4e6db22905c11d146f0f P 3d19d237cda549d982e93b80f2f7d6c9: Deleting consensus metadata
I20260812 11:17:09.071394 19154 catalog_manager.cc:4841] TS 3d19d237cda549d982e93b80f2f7d6c9 (127.30.255.131:41671): tablet 09fb14c50bff4e6db22905c11d146f0f (table test-table [id=d43c86304e7a47429f791b7fdc0b7a29]) successfully deleted
I20260812 11:17:09.073707 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 19568
W20260812 11:17:09.073690 19154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:38963 (error 108)
I20260812 11:17:09.076866 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:09.076583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38963 (local address 127.0.0.1:57670)
0812 11:17:09.076729 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:09.076789 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 11:17:09.076996 19154 catalog_manager.cc:4584] TS fa348dbf30dd4d44b28a51de5a3ddeb7 (127.30.255.129:38963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09fb14c50bff4e6db22905c11d146f0f: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38963: connect: Connection refused (error 111)
I20260812 11:17:09.088702 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 19840
I20260812 11:17:09.100453 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 19127
2026-08-12T11:17:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2713 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:17:09 dist-test-slave-gjrz krb5kdc[20161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:17:09 dist-test-slave-gjrz krb5kdc[20161](info): set up 2 sockets
Aug 12 11:17:09 dist-test-slave-gjrz krb5kdc[20161](info): commencing operation
krb5kdc: starting...
W20260812 11:17:10.295451 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:17:10 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:17:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangeradmin_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangeradmin_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangerlookup_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangerlookup_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
I20260812 11:17:10.449514 31742 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoRXcNR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/postgres -l logfile start
2026-08-12 11:17:12.681 UTC [20213] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 11:17:12.682 UTC [20213] LOG: listening on IPv4 address "127.30.255.148", port 45543
2026-08-12 11:17:12.684 UTC [20213] LOG: listening on Unix socket "/tmp/.s.PGSQL.45543"
2026-08-12 11:17:12.704 UTC [20217] LOG: database system was shut down at 2026-08-12 11:17:11 UTC
2026-08-12 11:17:12.707 UTC [20213] LOG: database system is ready to accept connections
I20260812 11:17:13.812319 31742 mini_postgres.cc:96] Postgres bound to 45543
2026-08-12 11:17:13.820 UTC [20230] FATAL: database "slave" does not exist
127.30.255.148:45543 - accepting connections
I20260812 11:17:13.821058 31742 mini_ranger.cc:161] Starting Ranger...
I20260812 11:17:13.836517 31742 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 11:17:14.248431 31742 mini_ranger.cc:87] Created ranger Postgres database
I20260812 11:17:14.248517 31742 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin
2026-08-12 11:17:14,603 [I] DB FLAVOR :POSTGRES
2026-08-12 11:17:14,603 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:17:14,603 [I] Checking connection..
2026-08-12 11:17:14,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 11:17:14,862 [I] Checking connection passed.
2026-08-12 11:17:14,862 [I] --------- Verifying version history table ---------
2026-08-12 11:17:14,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:17:15,126 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:17:15,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:17:15,359 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:17:15,359 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 11:17:15,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 11:17:15.604 UTC [20353] WARNING: there is no transaction in progress
2026-08-12 11:17:15,610 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 11:17:15,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:17:15,857 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 11:17:15,857 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:17:15,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 11:17:16,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 11:17:16,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-12 11:17:16,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:17:16,798 [I] Table x_portal_user does not exist in database ranger
2026-08-12 11:17:16,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:17:17,046 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 11:17:17,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:17:17,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:17:17,539 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 11:17:17,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 11:17:18.413 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.428 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.443 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.457 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.471 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.486 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.545 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.551 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.563 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.572 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.581 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.593 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.602 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.607 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.613 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:18.618 UTC [20563] WARNING: there is no transaction in progress
2026-08-12 11:17:19,154 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 11:17:19,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 11:17:19,396 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 11:17:19,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:17:19,630 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 11:17:19,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:17:19,872 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 11:17:19,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:17:20,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:17:20,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-12 11:17:20,575 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 11:17:20,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 11:17:20,808 [I] DB_PATCHES have already been applied
I20260812 11:17:20.813446 31742 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoRXcNR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 11:17:20.813512 31742 mini_ranger.cc:193] Using host: 127.30.255.148
I20260812 11:17:20.817724 31742 mini_ranger.cc:227] Ranger admin URL: http://127.30.255.148:53875
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 11:17:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 11:17:22.066561 31742 mini_ranger.cc:160] Time spent starting Ranger: real 8.246s user 0.001s sys 0.010s
Aug 12, 2026 11:17:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 11:17:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 11:17:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 11:17:41 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 11:17:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 11:17:42.974185 31742 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangerkms_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/rangerkms_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.255.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.255.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 11:17:43 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 11:17:43.073541 31742 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 11:17:43.084391 31742 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 11:17:43.871186 31742 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 11:17:43.871296 31742 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms
I20260812 11:17:43.871385 31742 mini_ranger_kms.cc:227] Using postgres at 127.30.255.148:45543
2026-08-12 11:17:44,138 [I] DB FLAVOR :POSTGRES
2026-08-12 11:17:44,138 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:17:44,138 [I] Checking connection
2026-08-12 11:17:44,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 11:17:44,378 [I] connection success
2026-08-12 11:17:44,378 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 11:17:44,378 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 11:17:44,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 11:17:44,613 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 11:17:44,614 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:17:44,614 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 11:17:44,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:45543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 11:17:44,875 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 11:17:45.335155 31742 mini_ranger_kms.cc:310] Created kms service
I20260812 11:17:45.533684 31742 mini_ranger_kms.cc:326] Created kudu user
I20260812 11:17:45.583623 31742 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 11:17:45 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:45.955224 31742 mini_ranger_kms.cc:384] Added ranger policy
I20260812 11:17:45.955410 31742 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 11:17:45.955438 31742 mini_ranger_kms.cc:240] Using host: 127.30.255.148
I20260812 11:17:45.960511 31742 mini_ranger_kms.cc:276] Ranger KMS PID: 20931
I20260812 11:17:45.960647 31742 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.255.148:59295
Aug 12, 2026 11:17:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:17:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:17:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 11:17:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 11:17:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 11:17:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 11:17:48 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 11:17:48 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:48.629817 31742 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.556s user 0.006s sys 0.245s
I20260812 11:17:48.629951 31742 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 11:17:48.630049 31742 mini_ranger_kms.cc:401] 127.30.255.148:59295/kms/v1/keys
Aug 12 11:17:48 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
Aug 12 11:17:51 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:17:51.164458 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33785
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:33785
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:59295/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:51.302591 21034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:51.302834 21034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:51.302888 21034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:51.306923 21034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:17:51.306974 21034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:51.307000 21034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:51.307022 21034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:17:51.307049 21034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:17:51.312215 21034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.255.148:59295/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:33785
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:33785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:51.313136 21034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:51.314043 21034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:51.323467 21046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:51.323488 21042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:51.323549 21034 server_base.cc:1034] running on GCE node
W20260812 11:17:51.323843 21043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:51.324347 21034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:51.325366 21034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:51.326534 21034 hybrid_clock.cc:648] HybridClock initialized: now 1786533471326503 us; error 50 us; skew 500 ppm
Aug 12 11:17:51 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533471, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:51.333396 21034 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:17:51.334306 21034 webserver.cc:460] Webserver started at http://127.30.255.190:44277/ using document root <none> and password file <none>
I20260812 11:17:51.334790 21034 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:51.334846 21034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:51.335049 21034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:17:51 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533471, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:51.381734 21034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "5477a40653bc4a25982749373c95ba7b"
format_stamp: "Formatted at 2026-08-12 11:17:51 on dist-test-slave-gjrz"
server_key: "598b8e5354e670182e459b60a38530b7"
server_key_iv: "ba6fa8f448937d3f80266f994b094051"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:51.382334 21034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "5477a40653bc4a25982749373c95ba7b"
format_stamp: "Formatted at 2026-08-12 11:17:51 on dist-test-slave-gjrz"
server_key: "598b8e5354e670182e459b60a38530b7"
server_key_iv: "ba6fa8f448937d3f80266f994b094051"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:51.386178 21034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 11:17:51.398828 21062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:51.400112 21034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:17:51.400233 21034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "5477a40653bc4a25982749373c95ba7b"
format_stamp: "Formatted at 2026-08-12 11:17:51 on dist-test-slave-gjrz"
server_key: "598b8e5354e670182e459b60a38530b7"
server_key_iv: "ba6fa8f448937d3f80266f994b094051"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:51.400339 21034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:51.409380 21034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:51.411958 21034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:51.412164 21034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:51.420279 21034 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:33785
I20260812 11:17:51.420324 21167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:33785 every 8 connection(s)
I20260812 11:17:51.421195 21034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:17:51.421696 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 21034
I20260812 11:17:51.421823 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:17:51.422130 31742 external_mini_cluster.cc:1428] Setting key 73a1a4797ecc5a32046fb14a89af1a9d
I20260812 11:17:51.424373 21168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:51.431423 21168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: Bootstrap starting.
Aug 12 11:17:51 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:17:51.433876 21168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:51.434855 21168 log.cc:824] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:51.436939 21168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: No bootstrap required, opened a new log
I20260812 11:17:51.439020 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:51.423529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.30.255.190:33785)
0812 11:17:51.423973 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:51.423991 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:51.424042 (+ 51us) server_negotiation.cc:407] Connection header received
0812 11:17:51.424856 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:51.424878 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:51.425284 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:51.425702 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:51.426554 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:51.428109 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:51.429241 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:51.429950 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:51.430108 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:51.433093 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:51.433101 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:51.433117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:51.433149 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:51.435542 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:51.436019 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:51.436026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:51.436032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:51.436164 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:51.436459 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:51.436462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:51.436463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:51.436943 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:51.437697 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:51.437842 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 11:17:51.438108 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":130,"threads_started":1}
I20260812 11:17:51.439952 21168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } }
I20260812 11:17:51.440227 21168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:51.440299 21168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5477a40653bc4a25982749373c95ba7b, State: Initialized, Role: FOLLOWER
I20260812 11:17:51.440757 21168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } }
I20260812 11:17:51.440927 21168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:17:51.441012 21168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:17:51.441119 21168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:51.442135 21168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } }
I20260812 11:17:51.442437 21168 leader_election.cc:302] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5477a40653bc4a25982749373c95ba7b; no voters:
I20260812 11:17:51.442715 21168 leader_election.cc:288] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:17:51.442801 21176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:51.443032 21176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [term 1 LEADER]: Becoming Leader. State: Replica: 5477a40653bc4a25982749373c95ba7b, State: Running, Role: LEADER
I20260812 11:17:51.443399 21176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } }
I20260812 11:17:51.444024 21168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:17:51.444993 21180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5477a40653bc4a25982749373c95ba7b. Latest consensus state: current_term: 1 leader_uuid: "5477a40653bc4a25982749373c95ba7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } } }
I20260812 11:17:51.445114 21180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:51.445333 21180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5477a40653bc4a25982749373c95ba7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5477a40653bc4a25982749373c95ba7b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 33785 } } }
I20260812 11:17:51.445403 21180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b [sys.catalog]: This master's current role is: LEADER
I20260812 11:17:51.446251 21168 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:17:51.446728 21196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:17:51.453527 21196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:17:51.462073 21196 catalog_manager.cc:1347] Generated new cluster ID: a13be6ef4b5f46d48804bf3ed77ed57f
I20260812 11:17:51.462213 21196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:17:51.476184 21196 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:17:51.477481 21196 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:17:51.489298 21196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: Generated new TSK 0
I20260812 11:17:51.490200 21196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:17:52.186551 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:52.345109 21263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:52.345418 21263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:52.345479 21263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:52.349807 21263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:52.349859 21263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:52.349913 21263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:17:52.356292 21263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.255.148:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:52.357414 21263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:52.358593 21263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:52.369784 21281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:52.370214 21275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:52.370330 21276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:52.371310 21263 server_base.cc:1034] running on GCE node
I20260812 11:17:52.371707 21263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:52.372303 21263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:52.373462 21263 hybrid_clock.cc:648] HybridClock initialized: now 1786533472373437 us; error 43 us; skew 500 ppm
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:52.376145 21263 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:17:52.376998 21263 webserver.cc:460] Webserver started at http://127.30.255.129:32933/ using document root <none> and password file <none>
I20260812 11:17:52.377615 21263 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:52.377735 21263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:52.378000 21263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:52.398352 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "a3a3be0c5b8a4193a3b381c9aa99372b"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "d847fb40af15f0338d0790f4446cb764"
server_key_iv: "78e862ff5fc74f930f78de6847348f12"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.399140 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "a3a3be0c5b8a4193a3b381c9aa99372b"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "d847fb40af15f0338d0790f4446cb764"
server_key_iv: "78e862ff5fc74f930f78de6847348f12"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.404165 21263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 11:17:52.424044 21294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.425510 21263 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260812 11:17:52.425626 21263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "a3a3be0c5b8a4193a3b381c9aa99372b"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "d847fb40af15f0338d0790f4446cb764"
server_key_iv: "78e862ff5fc74f930f78de6847348f12"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.425733 21263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:52.456027 21263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:52.459631 21263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:52.460060 21263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:52.460773 21263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:52.462239 21263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:52.462327 21263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.462404 21263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:52.462448 21263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.472891 21263 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41789
I20260812 11:17:52.474013 21263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:17:52.476917 21513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41789 every 8 connection(s)
I20260812 11:17:52.479947 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 21263
I20260812 11:17:52.480067 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:17:52.480332 31742 external_mini_cluster.cc:1428] Setting key f26dd16a853fda19a72dbade6e469d4e
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:17:52.496161 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:52.479106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33785 (local address 127.30.255.129:57457)
0812 11:17:52.481921 (+ 2815us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:52.481988 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:52.482953 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:52.483750 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:52.483759 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:52.484082 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:52.484699 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:52.484718 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.485900 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.485904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:52.487383 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:52.487396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.488137 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.488141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:52.488333 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:52.488962 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:52.488983 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:52.490647 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:52.493024 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:52.493029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:52.493044 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:52.493371 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:52.493707 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:52.493710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:52.493712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:52.493879 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:52.494331 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:52.494337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:52.494627 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 11:17:52.494822 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 11:17:52.495124 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2569,"thread_start_us":146,"threads_started":1}
I20260812 11:17:52.496865 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:52.478642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:57457 (local address 127.30.255.190:33785)
0812 11:17:52.479162 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:52.479169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:52.482096 (+ 2927us) server_negotiation.cc:407] Connection header received
0812 11:17:52.483316 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:52.483319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:52.483371 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:52.483447 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:52.484864 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.485742 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.487523 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.487879 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.487970 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:52.490972 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:52.490976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:52.490979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:52.491008 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:52.492893 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:52.493489 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:52.493493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:52.493495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:52.493599 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:52.493983 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:52.493987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:52.493989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:52.494196 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:52.494279 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:52.496571 (+ 2292us) server_negotiation.cc:299] Negotiation successful
0812 11:17:52.496699 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":121,"threads_started":1}
I20260812 11:17:52.497985 21515 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33785
I20260812 11:17:52.498205 21515 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:52.498711 21515 heartbeater.cc:511] Master 127.30.255.190:33785 requested a full tablet report, sending...
I20260812 11:17:52.500489 21088 ts_manager.cc:194] Registered new tserver with Master: a3a3be0c5b8a4193a3b381c9aa99372b (127.30.255.129:41789)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:17:52.502143 21088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:57457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:17:52.524328 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 11:17:52.646929 21532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:52.647176 21532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:52.647230 21532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:52.651234 21532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:52.651357 21532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:52.651453 21532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:17:52.656939 21532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.255.148:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:52.657848 21532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:52.658763 21532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:52.669034 21548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:52.669163 21546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:52.669339 21532 server_base.cc:1034] running on GCE node
W20260812 11:17:52.669466 21544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:52.669850 21532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:52.670446 21532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:52.671617 21532 hybrid_clock.cc:648] HybridClock initialized: now 1786533472671590 us; error 36 us; skew 500 ppm
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:52.674576 21532 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:17:52.675637 21532 webserver.cc:460] Webserver started at http://127.30.255.130:42657/ using document root <none> and password file <none>
I20260812 11:17:52.676340 21532 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:52.676442 21532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:52.676704 21532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:52.692214 21532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "3de1c8c3665a44ffbfc1d7de348b0a62"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "a23b62dc21557e5bcbec12b31fe90241"
server_key_iv: "48ac69192c4190e23cab9c359f973493"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.692920 21532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "3de1c8c3665a44ffbfc1d7de348b0a62"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "a23b62dc21557e5bcbec12b31fe90241"
server_key_iv: "48ac69192c4190e23cab9c359f973493"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.697202 21532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:17:52.717216 21565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.718796 21532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:17:52.718936 21532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "3de1c8c3665a44ffbfc1d7de348b0a62"
format_stamp: "Formatted at 2026-08-12 11:17:52 on dist-test-slave-gjrz"
server_key: "a23b62dc21557e5bcbec12b31fe90241"
server_key_iv: "48ac69192c4190e23cab9c359f973493"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:52.719067 21532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:52.767535 21532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:52.771165 21532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:52.771410 21532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:52.772133 21532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:52.773241 21532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:52.773375 21532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.773504 21532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:52.773586 21532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:52.788120 21532 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:38655
I20260812 11:17:52.789295 21532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:17:52.792644 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 21532
I20260812 11:17:52.792780 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:17:52.793103 31742 external_mini_cluster.cc:1428] Setting key 881148f60b7f5471e1c6389935c3286b
I20260812 11:17:52.790633 21798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:38655 every 8 connection(s)
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Aug 12 11:17:52 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:17:52.823870 21808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:52.793214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33785 (local address 127.30.255.130:46889)
0812 11:17:52.799367 (+ 6153us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:52.799427 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:52.800523 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:52.801289 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:52.801298 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:52.801660 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:52.802319 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:52.802338 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.804313 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.804317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:52.805728 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:52.805740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.812058 (+ 6318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.812062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:52.812264 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:52.812865 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:52.812886 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:52.818130 (+ 5244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:52.820155 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:52.820161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:52.820488 (+ 327us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:52.820841 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:52.821153 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:52.821157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:52.821159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:52.821333 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:52.821834 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:52.821841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:52.822178 (+ 337us) client_negotiation.cc:769] Sending connection context
0812 11:17:52.822438 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 11:17:52.822784 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5894,"thread_start_us":145,"threads_started":1}
I20260812 11:17:52.824582 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:52.794277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:46889 (local address 127.30.255.190:33785)
0812 11:17:52.794395 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:52.794399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:52.799540 (+ 5141us) server_negotiation.cc:407] Connection header received
0812 11:17:52.800774 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:52.800778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:52.800832 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:52.800885 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:52.803312 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.804155 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.811373 (+ 7218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:52.811763 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:52.811845 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:52.818332 (+ 6487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:52.818336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:52.818338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:52.818368 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:52.820024 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:52.820964 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:52.820967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:52.820969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:52.821058 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:52.821444 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:52.821447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:52.821449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:52.821681 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:52.821775 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:52.824277 (+ 2502us) server_negotiation.cc:299] Negotiation successful
0812 11:17:52.824415 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:17:52.826256 21799 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33785
I20260812 11:17:52.826509 21799 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:52.827075 21799 heartbeater.cc:511] Master 127.30.255.190:33785 requested a full tablet report, sending...
I20260812 11:17:52.829243 21088 ts_manager.cc:194] Registered new tserver with Master: 3de1c8c3665a44ffbfc1d7de348b0a62 (127.30.255.130:38655)
I20260812 11:17:52.829972 21088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:46889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:17:52.847030 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:17:53.010159 21811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:17:53.010553 21811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:17:53.010664 21811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:17:53.016543 21811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:17:53.017009 21811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:17:53.017159 21811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:17:53.024560 21811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.255.148:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:17:53.026027 21811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:17:53.027339 21811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:17:53.039714 21826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:53.040295 21825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:17:53.040546 21830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:17:53.041608 21811 server_base.cc:1034] running on GCE node
I20260812 11:17:53.042189 21811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:17:53.042874 21811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:17:53.044023 21811 hybrid_clock.cc:648] HybridClock initialized: now 1786533473043956 us; error 83 us; skew 500 ppm
Aug 12 11:17:53 dist-test-slave-gjrz krb5kdc[20161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533473, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:17:53.047623 21811 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:17:53.048610 21811 webserver.cc:460] Webserver started at http://127.30.255.131:35775/ using document root <none> and password file <none>
I20260812 11:17:53.049273 21811 fs_manager.cc:359] Metadata directory not provided
I20260812 11:17:53.049335 21811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:17:53.049544 21811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:17:53 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533473, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:53.073114 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "72bfb9a0663145d5989efe5b3373d916"
format_stamp: "Formatted at 2026-08-12 11:17:53 on dist-test-slave-gjrz"
server_key: "f91865d9b0fc82d750de0c977c922ce7"
server_key_iv: "0389d42474e1481c0bdec79a8c78ae30"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:53.087808 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "72bfb9a0663145d5989efe5b3373d916"
format_stamp: "Formatted at 2026-08-12 11:17:53 on dist-test-slave-gjrz"
server_key: "f91865d9b0fc82d750de0c977c922ce7"
server_key_iv: "0389d42474e1481c0bdec79a8c78ae30"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:53.102973 21811 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.005s sys 0.000s
Aug 12 11:17:53 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:17:53.121775 21843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:53.123461 21811 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260812 11:17:53.123590 21811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "72bfb9a0663145d5989efe5b3373d916"
format_stamp: "Formatted at 2026-08-12 11:17:53 on dist-test-slave-gjrz"
server_key: "f91865d9b0fc82d750de0c977c922ce7"
server_key_iv: "0389d42474e1481c0bdec79a8c78ae30"
server_key_version: "kuduclusterkey@0"
I20260812 11:17:53.123705 21811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:17:53.150175 21811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:17:53.154889 21811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:17:53.155344 21811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:17:53.156046 21811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:17:53.157086 21811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:17:53.157189 21811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:53.157284 21811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:17:53.157342 21811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:17:53.168318 21811 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:42053
I20260812 11:17:53.169386 21811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:17:53.171885 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:42053 every 8 connection(s)
I20260812 11:17:53.175885 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 21811
I20260812 11:17:53.176026 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:17:53.176332 31742 external_mini_cluster.cc:1428] Setting key d3324ff39ad6a8fd7af426bd56b806cd
Aug 12 11:17:53 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533473, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:17:53.189723 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.175303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:53565 (local address 127.30.255.190:33785)
0812 11:17:53.175435 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.175440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.175932 (+ 492us) server_negotiation.cc:407] Connection header received
0812 11:17:53.177125 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.177128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.177182 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.177229 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:53.178848 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.179787 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.181604 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.181978 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.182052 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.185051 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:53.185055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:53.185057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:53.185085 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:53.186745 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:53.187364 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:53.187368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:53.187370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:53.187457 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:53.187848 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:53.187851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:53.187853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:53.188065 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:53.188142 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.189415 (+ 1273us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.189557 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:17:53.192047 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.174164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:33785 (local address 127.30.255.131:53565)
0812 11:17:53.175779 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.175836 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.176902 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.177652 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.177660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.177979 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:17:53.178676 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.178700 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.179942 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.179946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.181447 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.181459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.182249 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.182252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.182455 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.183084 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:17:53.183104 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:17:53.184867 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:17:53.186884 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:53.186890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:53.186904 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:53.187221 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:53.187577 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:17:53.187580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:17:53.187582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:17:53.187746 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:17:53.188208 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:17:53.188214 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:17:53.189136 (+ 922us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.190716 (+ 1580us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.191042 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1398,"thread_start_us":144,"threads_started":1}
I20260812 11:17:53.193456 22104 heartbeater.cc:347] Connected to a master server at 127.30.255.190:33785
I20260812 11:17:53.193691 22104 heartbeater.cc:464] Registering TS with master...
I20260812 11:17:53.194211 22104 heartbeater.cc:511] Master 127.30.255.190:33785 requested a full tablet report, sending...
I20260812 11:17:53.195308 21088 ts_manager.cc:194] Registered new tserver with Master: 72bfb9a0663145d5989efe5b3373d916 (127.30.255.131:42053)
I20260812 11:17:53.196024 21088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:53565
I20260812 11:17:53.201510 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:17:53.213546 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.204047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.30.255.190:33785)
0812 11:17:53.204156 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.204160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.204760 (+ 600us) server_negotiation.cc:407] Connection header received
0812 11:17:53.204931 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.204934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.204976 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.205023 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:17:53.205941 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.206782 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.208045 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.208380 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.208447 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.210116 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:17:53.210120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:17:53.210123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:17:53.210148 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:17:53.211767 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:53.212246 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:53.212250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:53.212252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:53.212332 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:17:53.212633 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:17:53.212637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:17:53.212638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:17:53.212834 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:17:53.212933 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.213273 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.213384 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:17:53.217043 21088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:17:53.221271 21088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:17:53.251951 22139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.232343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41789 (local address 127.0.0.1:41044)
0812 11:17:53.234813 (+ 2470us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.234857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.234975 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.237184 (+ 2209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.237191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.237218 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:53.237348 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.237355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.238852 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.238855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.246533 (+ 7678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.246545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.251391 (+ 4846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.251395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.251520 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.251561 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.251673 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.251795 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2308,"thread_start_us":94,"threads_started":1}
I20260812 11:17:53.255097 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.234319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.30.255.131:42053)
0812 11:17:53.234634 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.234646 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.240563 (+ 5917us) server_negotiation.cc:407] Connection header received
0812 11:17:53.241740 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.241746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.241949 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.242065 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:53.242470 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.244384 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.252358 (+ 7974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.253229 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.253327 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.253464 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.254814 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.254932 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260812 11:17:53.257210 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.233358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:38655 (local address 127.0.0.1:52510)
0812 11:17:53.235476 (+ 2118us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.235490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.235582 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.241366 (+ 5784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.241369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.241385 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:53.241480 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.241486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.247997 (+ 6511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.248735 (+ 738us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.249963 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.249975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.256801 (+ 6826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.256805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.256905 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.256929 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.257002 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.257075 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2039,"thread_start_us":100,"threads_started":1}
I20260812 11:17:53.257829 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.233966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:42053 (local address 127.0.0.1:47348)
0812 11:17:53.240461 (+ 6495us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.240475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.241593 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.242226 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.242229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.242245 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:53.242337 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.242343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.244531 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.244534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.245798 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.245810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.254264 (+ 8454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.254268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.254370 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.254394 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.257619 (+ 3225us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.257697 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6422,"thread_start_us":76,"threads_started":1}
I20260812 11:17:53.259318 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.232483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.30.255.129:41789)
0812 11:17:53.235803 (+ 3320us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.235810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.235832 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:17:53.235907 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.235913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.236116 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.236265 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:53.237504 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.238713 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.250127 (+ 11414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.251032 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.251160 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.251325 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.259036 (+ 7711us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.259156 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3221,"thread_start_us":146,"threads_started":1}
I20260812 11:17:53.261330 21973 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee087bba4274abb92f5cbca7f037273 (DEFAULT_TABLE table=test-table [id=8c4135f2295149cf8214842d800a7fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:53.262511 21973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee087bba4274abb92f5cbca7f037273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:53.262408 21393 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee087bba4274abb92f5cbca7f037273 (DEFAULT_TABLE table=test-table [id=8c4135f2295149cf8214842d800a7fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:53.263474 21393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee087bba4274abb92f5cbca7f037273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:53.272234 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.233536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.30.255.130:38655)
0812 11:17:53.240761 (+ 7225us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.240780 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.240803 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:17:53.240877 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.240883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.241079 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.241225 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:53.246688 (+ 5463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.247861 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.255586 (+ 7725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.256475 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.256605 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.256732 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.271959 (+ 15227us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.272086 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7118,"thread_start_us":90,"threads_started":1}
I20260812 11:17:53.275599 22144 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Bootstrap starting.
I20260812 11:17:53.275236 21662 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee087bba4274abb92f5cbca7f037273 (DEFAULT_TABLE table=test-table [id=8c4135f2295149cf8214842d800a7fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:17:53.276364 21662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee087bba4274abb92f5cbca7f037273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:17:53.277998 22144 tablet_bootstrap.cc:654] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:53.278906 22144 log.cc:824] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:53.279678 22141 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Bootstrap starting.
I20260812 11:17:53.282420 22141 tablet_bootstrap.cc:654] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:53.283218 22141 log.cc:824] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:53.285344 22144 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: No bootstrap required, opened a new log
I20260812 11:17:53.285555 22144 ts_tablet_manager.cc:1397] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260812 11:17:53.290200 22144 raft_consensus.cc:348] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.290606 22144 raft_consensus.cc:374] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:53.290721 22144 raft_consensus.cc:729] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a3be0c5b8a4193a3b381c9aa99372b, State: Initialized, Role: FOLLOWER
I20260812 11:17:53.291294 22144 consensus_queue.cc:260] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.292621 22144 ts_tablet_manager.cc:1428] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260812 11:17:53.293082 21515 heartbeater.cc:503] Master 127.30.255.190:33785 was elected leader, sending a full tablet report...
I20260812 11:17:53.293548 22146 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Bootstrap starting.
I20260812 11:17:53.295786 22146 tablet_bootstrap.cc:654] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Neither blocks nor log segments found. Creating new log.
I20260812 11:17:53.297214 22146 log.cc:824] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Log is configured to *not* fsync() on all Append() calls
I20260812 11:17:53.299652 22141 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: No bootstrap required, opened a new log
I20260812 11:17:53.299853 22141 ts_tablet_manager.cc:1397] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260812 11:17:53.304059 22141 raft_consensus.cc:348] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.304383 22141 raft_consensus.cc:374] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:53.304508 22141 raft_consensus.cc:729] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72bfb9a0663145d5989efe5b3373d916, State: Initialized, Role: FOLLOWER
I20260812 11:17:53.305042 22141 consensus_queue.cc:260] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.306061 22141 ts_tablet_manager.cc:1428] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 11:17:53.306447 22104 heartbeater.cc:503] Master 127.30.255.190:33785 was elected leader, sending a full tablet report...
I20260812 11:17:53.308923 22146 tablet_bootstrap.cc:492] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: No bootstrap required, opened a new log
I20260812 11:17:53.309175 22146 ts_tablet_manager.cc:1397] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260812 11:17:53.313092 22146 raft_consensus.cc:348] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.313398 22146 raft_consensus.cc:374] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:17:53.313486 22146 raft_consensus.cc:729] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de1c8c3665a44ffbfc1d7de348b0a62, State: Initialized, Role: FOLLOWER
I20260812 11:17:53.313987 22146 consensus_queue.cc:260] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.314945 22146 ts_tablet_manager.cc:1428] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:17:53.315326 21799 heartbeater.cc:503] Master 127.30.255.190:33785 was elected leader, sending a full tablet report...
W20260812 11:17:53.434330 22109 tablet.cc:2369] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 11:17:53 dist-test-slave-gjrz krb5kdc[20161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533472, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
W20260812 11:17:53.479815 21517 tablet.cc:2369] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:17:53.542186 21805 tablet.cc:2369] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:17:53.555320 22152 raft_consensus.cc:482] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:17:53.555572 22152 raft_consensus.cc:504] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.557031 22152 leader_election.cc:288] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a3be0c5b8a4193a3b381c9aa99372b (127.30.255.129:41789), 3de1c8c3665a44ffbfc1d7de348b0a62 (127.30.255.130:38655)
I20260812 11:17:53.563033 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.557464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41789 (local address 127.30.255.131:33549)
0812 11:17:53.557615 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.557633 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.557735 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.559092 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.559096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.559119 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:53.559241 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.559249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.560342 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.560345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.561596 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.561608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.562617 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.562621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.562724 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.562750 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.562823 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.562899 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 11:17:53.564079 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.557867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:33549 (local address 127.30.255.129:41789)
0812 11:17:53.557992 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.557997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.558013 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:17:53.558071 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.558074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.558124 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.558205 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:53.559407 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.560201 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.561749 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.562369 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.562489 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.562544 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.563872 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.563949 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:17:53.565258 21430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee087bba4274abb92f5cbca7f037273" candidate_uuid: "72bfb9a0663145d5989efe5b3373d916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" is_pre_election: true
I20260812 11:17:53.565613 21430 raft_consensus.cc:2454] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72bfb9a0663145d5989efe5b3373d916 in term 0.
I20260812 11:17:53.566334 21847 leader_election.cc:302] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72bfb9a0663145d5989efe5b3373d916, a3a3be0c5b8a4193a3b381c9aa99372b; no voters:
I20260812 11:17:53.566723 22152 raft_consensus.cc:2781] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:17:53.566813 22152 raft_consensus.cc:482] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:17:53.566859 22152 raft_consensus.cc:3037] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:53.568356 22152 raft_consensus.cc:504] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.568759 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.558496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:38655 (local address 127.30.255.131:40701)
0812 11:17:53.563180 (+ 4684us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.563193 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:17:53.563294 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:17:53.563683 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:17:53.563686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:17:53.563707 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:17:53.563785 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.563791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.566190 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.566193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.567374 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:17:53.567385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.568381 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.568384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:17:53.568474 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.568494 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 11:17:53.568552 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:17:53.568612 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4631,"thread_start_us":133,"threads_started":1}
I20260812 11:17:53.569113 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.558914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:40701 (local address 127.30.255.130:38655)
0812 11:17:53.559029 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.559033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.563452 (+ 4419us) server_negotiation.cc:407] Connection header received
0812 11:17:53.563518 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.563521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.563569 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.563648 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:17:53.563932 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.564677 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.567520 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.568155 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.568245 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.568290 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.568910 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.569000 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:17:53.569726 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee087bba4274abb92f5cbca7f037273" candidate_uuid: "72bfb9a0663145d5989efe5b3373d916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" is_pre_election: true
I20260812 11:17:53.570006 21701 raft_consensus.cc:2454] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72bfb9a0663145d5989efe5b3373d916 in term 0.
I20260812 11:17:53.570339 21430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee087bba4274abb92f5cbca7f037273" candidate_uuid: "72bfb9a0663145d5989efe5b3373d916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b"
I20260812 11:17:53.570457 21430 raft_consensus.cc:3037] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:53.571688 21430 raft_consensus.cc:2454] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72bfb9a0663145d5989efe5b3373d916 in term 1.
I20260812 11:17:53.572018 22152 leader_election.cc:288] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [CANDIDATE]: Term 1 election: Requested vote from peers a3a3be0c5b8a4193a3b381c9aa99372b (127.30.255.129:41789), 3de1c8c3665a44ffbfc1d7de348b0a62 (127.30.255.130:38655)
I20260812 11:17:53.572060 21847 leader_election.cc:302] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72bfb9a0663145d5989efe5b3373d916, a3a3be0c5b8a4193a3b381c9aa99372b; no voters:
I20260812 11:17:53.572607 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee087bba4274abb92f5cbca7f037273" candidate_uuid: "72bfb9a0663145d5989efe5b3373d916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62"
I20260812 11:17:53.572654 22152 raft_consensus.cc:2781] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:17:53.572726 21701 raft_consensus.cc:3037] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:17:53.572855 22152 raft_consensus.cc:686] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 1 LEADER]: Becoming Leader. State: Replica: 72bfb9a0663145d5989efe5b3373d916, State: Running, Role: LEADER
I20260812 11:17:53.573179 22152 consensus_queue.cc:237] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } }
I20260812 11:17:53.573984 21701 raft_consensus.cc:2454] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72bfb9a0663145d5989efe5b3373d916 in term 1.
I20260812 11:17:53.576231 21088 catalog_manager.cc:5495] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72bfb9a0663145d5989efe5b3373d916 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "72bfb9a0663145d5989efe5b3373d916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72bfb9a0663145d5989efe5b3373d916" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 42053 } health_report { overall_health: HEALTHY } } }
I20260812 11:17:53.605692 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.600101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.30.255.131:42053)
0812 11:17:53.600213 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:17:53.600217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:17:53.600232 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:17:53.600398 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:17:53.600402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:17:53.600469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:17:53.600587 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:17:53.600998 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.601796 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.603227 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:17:53.603623 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:17:53.603725 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:17:53.604351 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:17:53.604915 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:17:53.605035 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:17:53.605463 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 11:17:53.605523 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:17:53.613019 21701 raft_consensus.cc:1261] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 1 FOLLOWER]: Refusing update from remote peer 72bfb9a0663145d5989efe5b3373d916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:17:53.613032 21430 raft_consensus.cc:1261] T 6ee087bba4274abb92f5cbca7f037273 P a3a3be0c5b8a4193a3b381c9aa99372b [term 1 FOLLOWER]: Refusing update from remote peer 72bfb9a0663145d5989efe5b3373d916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:17:53.613782 22163 consensus_queue.cc:1035] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de1c8c3665a44ffbfc1d7de348b0a62" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:17:53.613966 22152 consensus_queue.cc:1035] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a3be0c5b8a4193a3b381c9aa99372b" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:17:53.625033 22173 mvcc.cc:205] Tried to move back new op lower bound from 7317641107909562368 to 7317641107758514176. Current Snapshot: MvccSnapshot[applied={T|T < 7317641107909562368}]
I20260812 11:17:53.629554 22179 mvcc.cc:205] Tried to move back new op lower bound from 7317641107909562368 to 7317641107758514176. Current Snapshot: MvccSnapshot[applied={T|T < 7317641107909562368}]
I20260812 11:17:53.637717 21084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:17:53.637866 21084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:17:53.640326 21084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5477a40653bc4a25982749373c95ba7b: Sending DeleteTablet for 3 replicas of tablet 6ee087bba4274abb92f5cbca7f037273
I20260812 11:17:53.641067 21662 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee087bba4274abb92f5cbca7f037273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:53 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:52510
I20260812 11:17:53.641520 22204 tablet_replica.cc:330] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: stopping tablet replica
I20260812 11:17:53.641527 21393 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee087bba4274abb92f5cbca7f037273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:53 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:41044
I20260812 11:17:53.641803 22204 raft_consensus.cc:2229] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:17:53.642078 22204 raft_consensus.cc:2258] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:17:53.642266 21973 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee087bba4274abb92f5cbca7f037273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:17:53 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47348
I20260812 11:17:53.643437 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 21263
I20260812 11:17:53.657857 22204 ts_tablet_manager.cc:1905] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:17:53.659475 22208 tablet_replica.cc:330] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: stopping tablet replica
I20260812 11:17:53.659750 22208 raft_consensus.cc:2229] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:17:53.660116 22208 raft_consensus.cc:2258] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:17:53.660760 22204 ts_tablet_manager.cc:1918] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:17:53.660921 22204 log.cc:1196] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/6ee087bba4274abb92f5cbca7f037273
I20260812 11:17:53.661324 22204 ts_tablet_manager.cc:1939] T 6ee087bba4274abb92f5cbca7f037273 P 3de1c8c3665a44ffbfc1d7de348b0a62: Deleting consensus metadata
I20260812 11:17:53.661526 22208 ts_tablet_manager.cc:1905] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:17:53.662650 21066 catalog_manager.cc:4841] TS 3de1c8c3665a44ffbfc1d7de348b0a62 (127.30.255.130:38655): tablet 6ee087bba4274abb92f5cbca7f037273 (table test-table [id=8c4135f2295149cf8214842d800a7fcd]) successfully deleted
I20260812 11:17:53.664239 22208 ts_tablet_manager.cc:1918] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:17:53.664332 22208 log.cc:1196] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/6ee087bba4274abb92f5cbca7f037273
I20260812 11:17:53.664665 22208 ts_tablet_manager.cc:1939] T 6ee087bba4274abb92f5cbca7f037273 P 72bfb9a0663145d5989efe5b3373d916: Deleting consensus metadata
I20260812 11:17:53.665758 21065 catalog_manager.cc:4841] TS 72bfb9a0663145d5989efe5b3373d916 (127.30.255.131:42053): tablet 6ee087bba4274abb92f5cbca7f037273 (table test-table [id=8c4135f2295149cf8214842d800a7fcd]) successfully deleted
W20260812 11:17:53.667469 21064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:41789 (error 108)
I20260812 11:17:53.668699 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:17:53.668387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41789 (local address 127.0.0.1:41046)
0812 11:17:53.668551 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 11:17:53.668615 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 11:17:53.668934 21064 catalog_manager.cc:4584] TS a3a3be0c5b8a4193a3b381c9aa99372b (127.30.255.129:41789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ee087bba4274abb92f5cbca7f037273: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41789: connect: Connection refused (error 111)
I20260812 11:17:53.671568 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 21532
I20260812 11:17:53.685521 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 21811
I20260812 11:17:53.699667 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 21034
I20260812 11:17:53.725661 31742 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 11:17:54.069340 31742 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 11:17:54.069458 31742 mini_ranger.cc:66] Stopping Ranger...
I20260812 11:17:54.413000 31742 mini_ranger.cc:68] Stopped Ranger
2026-08-12 11:17:54.413 UTC [20213] LOG: received smart shutdown request
2026-08-12 11:17:54.416 UTC [20213] LOG: background worker "logical replication launcher" (PID 20227) exited with exit code 1
2026-08-12 11:17:54.417 UTC [20218] LOG: shutting down
2026-08-12 11:17:54.541 UTC [20213] LOG: database system is shut down
2026-08-12T11:17:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45526 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:17:54 dist-test-slave-gjrz krb5kdc[22229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:17:54 dist-test-slave-gjrz krb5kdc[22229](info): set up 2 sockets
Aug 12 11:17:54 dist-test-slave-gjrz krb5kdc[22229](info): commencing operation
krb5kdc: starting...
W20260812 11:17:55.834697 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:17:55 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:17:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:17:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangeradmin_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangeradmin_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangerlookup_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangerlookup_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
I20260812 11:17:55.983803 31742 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoRXcNR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/postgres -l logfile start
2026-08-12 11:17:58.099 UTC [22282] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 11:17:58.099 UTC [22282] LOG: listening on IPv4 address "127.30.255.148", port 53959
2026-08-12 11:17:58.102 UTC [22282] LOG: listening on Unix socket "/tmp/.s.PGSQL.53959"
2026-08-12 11:17:58.121 UTC [22286] LOG: database system was shut down at 2026-08-12 11:17:56 UTC
2026-08-12 11:17:58.125 UTC [22282] LOG: database system is ready to accept connections
I20260812 11:17:59.246765 31742 mini_postgres.cc:96] Postgres bound to 53959
2026-08-12 11:17:59.254 UTC [22304] FATAL: database "slave" does not exist
127.30.255.148:53959 - accepting connections
I20260812 11:17:59.254583 31742 mini_ranger.cc:161] Starting Ranger...
I20260812 11:17:59.269256 31742 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 11:17:59.640615 31742 mini_ranger.cc:87] Created ranger Postgres database
I20260812 11:17:59.640688 31742 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin
2026-08-12 11:17:59,909 [I] DB FLAVOR :POSTGRES
2026-08-12 11:17:59,910 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:17:59,910 [I] Checking connection..
2026-08-12 11:17:59,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 11:18:00,155 [I] Checking connection passed.
2026-08-12 11:18:00,156 [I] --------- Verifying version history table ---------
2026-08-12 11:18:00,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:18:00,405 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:18:00,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:18:00,645 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:18:00,645 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 11:18:00,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 11:18:00.891 UTC [22441] WARNING: there is no transaction in progress
2026-08-12 11:18:00,897 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 11:18:00,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:18:01,132 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 11:18:01,132 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:18:01,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 11:18:01,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 11:18:01,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-12 11:18:01,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:18:02,102 [I] Table x_portal_user does not exist in database ranger
2026-08-12 11:18:02,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:18:02,346 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 11:18:02,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:18:02,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:18:02,849 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 11:18:02,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 11:18:03.712 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.726 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.741 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.754 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.771 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.788 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.836 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.842 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.855 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.864 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.873 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.885 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.894 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.899 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.905 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:03.911 UTC [22655] WARNING: there is no transaction in progress
2026-08-12 11:18:04,474 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 11:18:04,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 11:18:04,698 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 11:18:04,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:18:04,942 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 11:18:04,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:18:05,187 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 11:18:05,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:18:05,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:18:05,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-12 11:18:05,938 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 11:18:05,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 11:18:06,189 [I] DB_PATCHES have already been applied
I20260812 11:18:06.195348 31742 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoRXcNR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 11:18:06.195407 31742 mini_ranger.cc:193] Using host: 127.30.255.148
I20260812 11:18:06.199529 31742 mini_ranger.cc:227] Ranger admin URL: http://127.30.255.148:36729
Aug 12, 2026 11:18:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:18:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:18:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 11:18:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 11:18:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 11:18:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 11:18:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 11:18:07.480769 31742 mini_ranger.cc:160] Time spent starting Ranger: real 8.226s user 0.000s sys 0.010s
Aug 12, 2026 11:18:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 11:18:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 11:18:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 11:18:26 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 11:18:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 11:18:28.342554 31742 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.255.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.255.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangerkms_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.255.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/rangerkms_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.255.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.255.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.255.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/HTTP_127.30.255.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 11:18:28 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 11:18:28.419113 31742 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 11:18:28.427979 31742 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 11:18:29.115249 31742 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 11:18:29.115355 31742 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms
I20260812 11:18:29.115370 31742 mini_ranger_kms.cc:227] Using postgres at 127.30.255.148:53959
2026-08-12 11:18:29,278 [I] DB FLAVOR :POSTGRES
2026-08-12 11:18:29,279 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:18:29,279 [I] Checking connection
2026-08-12 11:18:29,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 11:18:29,504 [I] connection success
2026-08-12 11:18:29,504 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 11:18:29,505 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 11:18:29,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 11:18:29,762 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 11:18:29,762 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:18:29,763 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 11:18:29,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.255.148:53959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 11:18:30,027 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 11:18:30.431618 31742 mini_ranger_kms.cc:310] Created kms service
I20260812 11:18:30.616863 31742 mini_ranger_kms.cc:326] Created kudu user
I20260812 11:18:30.659719 31742 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 11:18:30 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:31.016718 31742 mini_ranger_kms.cc:384] Added ranger policy
I20260812 11:18:31.016984 31742 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoRXcNR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoRXcNR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 11:18:31.017012 31742 mini_ranger_kms.cc:240] Using host: 127.30.255.148
I20260812 11:18:31.021682 31742 mini_ranger_kms.cc:276] Ranger KMS PID: 23003
I20260812 11:18:31.021768 31742 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.255.148:36883
Aug 12, 2026 11:18:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:18:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 11:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 11:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 11:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 11:18:33 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533513, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:33.681744 31742 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.263s user 0.009s sys 0.139s
I20260812 11:18:33.681872 31742 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 11:18:33.681977 31742 mini_ranger_kms.cc:401] 127.30.255.148:36883/kms/v1/keys
Aug 12 11:18:33 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533513, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
Aug 12 11:18:34 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533513, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.255.148@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
Aug 12 11:18:36 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:18:36.427774 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44089
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:44089
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:36.539371 23104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:36.539604 23104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:36.539645 23104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:36.543607 23104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:36.543660 23104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:36.543678 23104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:36.543692 23104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:36.543707 23104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:36.548992 23104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.255.148:36883/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:44089
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:36.549963 23104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:36.550922 23104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:36.559549 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:36.559656 23117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:36.559738 23104 server_base.cc:1034] running on GCE node
W20260812 11:18:36.559542 23119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:36.560348 23104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:36.561611 23104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:36.562803 23104 hybrid_clock.cc:648] HybridClock initialized: now 1786533516562772 us; error 41 us; skew 500 ppm
Aug 12 11:18:36 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:36.566006 23104 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:36.567021 23104 webserver.cc:460] Webserver started at http://127.30.255.190:37833/ using document root <none> and password file <none>
I20260812 11:18:36.567680 23104 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:36.567755 23104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:36.568024 23104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:18:36 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:36.615548 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "26cc4edc3c1a4df698a38a3a573b7378"
format_stamp: "Formatted at 2026-08-12 11:18:36 on dist-test-slave-gjrz"
server_key: "6956d72d6d3ce2cf1ae901bf31b3b6a9"
server_key_iv: "0cd84d654a8f6a3b4d19d75a987906d4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:36.616282 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "26cc4edc3c1a4df698a38a3a573b7378"
format_stamp: "Formatted at 2026-08-12 11:18:36 on dist-test-slave-gjrz"
server_key: "6956d72d6d3ce2cf1ae901bf31b3b6a9"
server_key_iv: "0cd84d654a8f6a3b4d19d75a987906d4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:36.621076 23104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:18:36.638533 23129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:36.640187 23104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:18:36.640327 23104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "26cc4edc3c1a4df698a38a3a573b7378"
format_stamp: "Formatted at 2026-08-12 11:18:36 on dist-test-slave-gjrz"
server_key: "6956d72d6d3ce2cf1ae901bf31b3b6a9"
server_key_iv: "0cd84d654a8f6a3b4d19d75a987906d4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:36.640472 23104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:36.677480 23104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:36.680626 23104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:36.680822 23104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:36.688552 23104 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:44089
I20260812 11:18:36.688587 23228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:44089 every 8 connection(s)
I20260812 11:18:36.689544 23104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:36.692950 23229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:36.694355 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 23104
I20260812 11:18:36.694478 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:18:36.694756 31742 external_mini_cluster.cc:1428] Setting key 437cfd074716c8e530c32b951b999c83
I20260812 11:18:36.700110 23229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Bootstrap starting.
Aug 12 11:18:36 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:36.702836 23229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:36.703787 23229 log.cc:824] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:36.706094 23229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: No bootstrap required, opened a new log
I20260812 11:18:36.708120 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:36.696136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.30.255.190:44089)
0812 11:18:36.696522 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:36.696538 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:36.696573 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:18:36.697207 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:36.697225 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:36.697537 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:36.697839 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:36.698615 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:36.699630 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:36.700483 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:36.700952 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:36.701104 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:36.703361 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:36.703367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:36.703380 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:36.703413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:36.705490 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:36.705887 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:36.705892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:36.705896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:36.705998 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:36.706283 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:36.706286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:36.706288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:36.706664 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:36.706856 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:36.707121 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 11:18:36.707419 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260812 11:18:36.709968 23229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:36.710269 23229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:36.710364 23229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26cc4edc3c1a4df698a38a3a573b7378, State: Initialized, Role: FOLLOWER
I20260812 11:18:36.710812 23229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:36.710968 23229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:36.711035 23229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:36.711128 23229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:36.712211 23229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:36.712589 23229 leader_election.cc:302] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26cc4edc3c1a4df698a38a3a573b7378; no voters:
I20260812 11:18:36.712939 23229 leader_election.cc:288] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:18:36.713014 23238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:36.713219 23238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 LEADER]: Becoming Leader. State: Replica: 26cc4edc3c1a4df698a38a3a573b7378, State: Running, Role: LEADER
I20260812 11:18:36.713738 23238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:36.714226 23229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:36.715368 23239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26cc4edc3c1a4df698a38a3a573b7378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } } }
I20260812 11:18:36.715570 23239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:36.715828 23239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26cc4edc3c1a4df698a38a3a573b7378. Latest consensus state: current_term: 1 leader_uuid: "26cc4edc3c1a4df698a38a3a573b7378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } } }
I20260812 11:18:36.715960 23239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:36.716778 23229 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:18:36.716778 23258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:36.722783 23258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:36.731667 23258 catalog_manager.cc:1347] Generated new cluster ID: 036d0214015d4b0599abe278a89fe1b9
I20260812 11:18:36.731794 23258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:36.751435 23258 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:18:36.753497 23258 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:36.762980 23258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Generated new TSK 0
I20260812 11:18:36.763857 23258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:18:37.454213 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:37.593411 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:37.593748 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:37.593818 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:37.598825 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:37.598898 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:37.598981 23314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:18:37.604964 23314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:37.605929 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:37.606890 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:37.618729 23335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:37.618738 23332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:37.618738 23328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:37.621826 23314 server_base.cc:1034] running on GCE node
I20260812 11:18:37.622257 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:37.622836 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:37.624028 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786533517623987 us; error 56 us; skew 500 ppm
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:37.627729 23314 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:18:37.628758 23314 webserver.cc:460] Webserver started at http://127.30.255.129:41539/ using document root <none> and password file <none>
I20260812 11:18:37.629421 23314 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:37.629519 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:37.629745 23314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:37.645994 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "91666987d4344ef4bb97a13b5c5aac61"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "6cd87619f7cac4f5aa0e0302af5ff13a"
server_key_iv: "e024c1443806e8ba8e3095b456b265d3"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:37.646809 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "91666987d4344ef4bb97a13b5c5aac61"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "6cd87619f7cac4f5aa0e0302af5ff13a"
server_key_iv: "e024c1443806e8ba8e3095b456b265d3"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:37.652175 23314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 11:18:37.682948 23351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:37.684734 23314 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.003s sys 0.000s
I20260812 11:18:37.684935 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "91666987d4344ef4bb97a13b5c5aac61"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "6cd87619f7cac4f5aa0e0302af5ff13a"
server_key_iv: "e024c1443806e8ba8e3095b456b265d3"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:37.685158 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:37.721350 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:37.725109 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:37.725365 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:37.726188 23314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:37.727233 23314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:37.727375 23314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:37.727496 23314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:37.727586 23314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:37.738251 23314 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:37431
I20260812 11:18:37.738400 23557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:37431 every 8 connection(s)
I20260812 11:18:37.739440 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:18:37.743562 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 23314
I20260812 11:18:37.743695 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:18:37.744014 31742 external_mini_cluster.cc:1428] Setting key 46f25c33dde0eedf802429288575db10
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:37.757211 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:37.741573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:41097 (local address 127.30.255.190:44089)
0812 11:18:37.741858 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:37.741866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:37.742806 (+ 940us) server_negotiation.cc:407] Connection header received
0812 11:18:37.744105 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:37.744109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:37.744170 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:37.744234 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:37.746578 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:37.747499 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:37.749225 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:37.749585 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:37.749661 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:37.752739 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:37.752743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:37.752746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:37.752774 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:37.754706 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:37.755439 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:37.755443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:37.755445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:37.755552 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:37.755948 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:37.755952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:37.755954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:37.756171 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:37.756252 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:37.756881 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 11:18:37.757007 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:18:37.759070 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:37.742073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.129:41097)
0812 11:18:37.742634 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:37.742695 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:37.743831 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:37.744984 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:37.744994 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:37.745555 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:37.746374 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:37.746398 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:37.747666 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:37.747669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:37.749079 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:37.749092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:37.749865 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:37.749869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:37.750110 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:37.750745 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:37.750765 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:37.752545 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:37.754846 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:37.754853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:37.754868 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:37.755320 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:37.755668 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:37.755671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:37.755673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:37.755838 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:37.756318 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:37.756324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:37.756617 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 11:18:37.757704 (+ 1087us) client_negotiation.cc:240] Negotiation successful
0812 11:18:37.758033 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:18:37.760536 23562 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:37.760766 23562 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:37.761307 23562 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:37.763463 23167 ts_manager.cc:194] Registered new tserver with Master: 91666987d4344ef4bb97a13b5c5aac61 (127.30.255.129:37431)
I20260812 11:18:37.764946 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:41097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:18:37.778210 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 11:18:37.936600 23572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:37.936918 23572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:37.936978 23572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:37.942821 23572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:37.942902 23572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:37.942983 23572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:18:37.950544 23572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:37.951777 23572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:37.952777 23572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:37.964926 23583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:37.965425 23584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:37.965267 23572 server_base.cc:1034] running on GCE node
W20260812 11:18:37.964906 23590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:37.965888 23572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:37.966470 23572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:37.967625 23572 hybrid_clock.cc:648] HybridClock initialized: now 1786533517967602 us; error 39 us; skew 500 ppm
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:37.970211 23572 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:18:37.971091 23572 webserver.cc:460] Webserver started at http://127.30.255.130:40615/ using document root <none> and password file <none>
I20260812 11:18:37.971673 23572 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:37.971740 23572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:37.971976 23572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:18:37 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:37.994098 23572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "5fe8635066df48cc84d355cf23871772"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "cd778e740a7c7c89fcc6857443f26625"
server_key_iv: "367b04e6ae9d4c92557032a2b7a7a7e4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:37.995121 23572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "5fe8635066df48cc84d355cf23871772"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "cd778e740a7c7c89fcc6857443f26625"
server_key_iv: "367b04e6ae9d4c92557032a2b7a7a7e4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.000669 23572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 11:18:38.020066 23601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.021695 23572 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260812 11:18:38.021903 23572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "5fe8635066df48cc84d355cf23871772"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "cd778e740a7c7c89fcc6857443f26625"
server_key_iv: "367b04e6ae9d4c92557032a2b7a7a7e4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.022116 23572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:38.055219 23572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:38.058087 23572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:38.058251 23572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:38.058831 23572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:38.059713 23572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:38.059762 23572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.059803 23572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:38.059830 23572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.069854 23572 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:45729
I20260812 11:18:38.069938 23842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:45729 every 8 connection(s)
I20260812 11:18:38.070755 23572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:18:38.076488 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 23572
I20260812 11:18:38.076615 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:18:38.076929 31742 external_mini_cluster.cc:1428] Setting key e75da45e205656a3d6ecaf5e69d84c0f
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:38.085623 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:38.072595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:39391 (local address 127.30.255.190:44089)
0812 11:18:38.072751 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:38.072755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:38.073614 (+ 859us) server_negotiation.cc:407] Connection header received
0812 11:18:38.074543 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:38.074547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:38.074611 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:38.074719 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:38.075835 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.076651 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.078103 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.078367 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.078458 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:38.081349 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:38.081353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:38.081356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:38.081384 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:38.083082 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:38.083724 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:38.083728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:38.083729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:38.083835 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:38.084270 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:38.084273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:38.084275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:38.084512 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:38.084612 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:38.085296 (+ 684us) server_negotiation.cc:299] Negotiation successful
0812 11:18:38.085438 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:18:38.087495 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:38.072971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.130:39391)
0812 11:18:38.073442 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:38.073496 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:38.074335 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:38.074864 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:38.074871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:38.075145 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:38.075680 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:38.075695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.076806 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.076810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:38.077973 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:38.077984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.078618 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.078623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:38.078827 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:38.079520 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:38.079541 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:38.081160 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:38.083213 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:38.083219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:38.083234 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:38.083595 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:38.083971 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:38.083975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:38.083977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:38.084158 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:38.084687 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:38.084694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:38.085001 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 11:18:38.086077 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0812 11:18:38.086421 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
I20260812 11:18:38.088986 23843 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:38.089226 23843 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:38.089689 23843 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:18:38.090770 23167 ts_manager.cc:194] Registered new tserver with Master: 5fe8635066df48cc84d355cf23871772 (127.30.255.130:45729)
I20260812 11:18:38.091411 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:39391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:18:38.110334 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
W20260812 11:18:38.277432 23853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:38.277757 23853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:38.277815 23853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:38.283771 23853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:38.283838 23853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:38.283917 23853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:18:38.291437 23853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:38.292872 23853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:38.294274 23853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:38.306703 23866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:38.306708 23869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:38.307199 23863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:38.308310 23853 server_base.cc:1034] running on GCE node
I20260812 11:18:38.308732 23853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:38.309427 23853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:38.312690 23853 hybrid_clock.cc:648] HybridClock initialized: now 1786533518312618 us; error 84 us; skew 500 ppm
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:38.316169 23853 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:18:38.317126 23853 webserver.cc:460] Webserver started at http://127.30.255.131:46175/ using document root <none> and password file <none>
I20260812 11:18:38.317759 23853 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:38.317823 23853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:38.318041 23853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:38.341145 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "07a8cbe6ded54581ab8235cf22183d46"
format_stamp: "Formatted at 2026-08-12 11:18:38 on dist-test-slave-gjrz"
server_key: "97dca960bd7df25266505f1fded15520"
server_key_iv: "e7ef888a0fc35387e4b446a9a774c2fd"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.359104 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "07a8cbe6ded54581ab8235cf22183d46"
format_stamp: "Formatted at 2026-08-12 11:18:38 on dist-test-slave-gjrz"
server_key: "97dca960bd7df25266505f1fded15520"
server_key_iv: "e7ef888a0fc35387e4b446a9a774c2fd"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.372754 23853 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.003s sys 0.000s
I20260812 11:18:38.398761 23880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.400424 23853 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.000s
I20260812 11:18:38.400559 23853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "07a8cbe6ded54581ab8235cf22183d46"
format_stamp: "Formatted at 2026-08-12 11:18:38 on dist-test-slave-gjrz"
server_key: "97dca960bd7df25266505f1fded15520"
server_key_iv: "e7ef888a0fc35387e4b446a9a774c2fd"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.400710 23853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:38.425253 23853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:38.428650 23853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:38.428828 23853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:38.429450 23853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:38.430401 23853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:38.430457 23853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.430511 23853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:38.430538 23853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.440810 23853 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:33549
I20260812 11:18:38.441843 23853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:18:38.445788 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 23853
I20260812 11:18:38.445910 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:18:38.446187 31742 external_mini_cluster.cc:1428] Setting key bdf6834a9757d8784c7a7535f4fb7f0a
I20260812 11:18:38.452390 24113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:33549 every 8 connection(s)
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:38.469775 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:38.454291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:50305 (local address 127.30.255.190:44089)
0812 11:18:38.454416 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:38.454420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:38.456162 (+ 1742us) server_negotiation.cc:407] Connection header received
0812 11:18:38.457379 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:38.457383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:38.457433 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:38.457479 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:38.460421 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.461196 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.462779 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.463090 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.463155 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:38.466005 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:38.466009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:38.466012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:38.466037 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:38.467598 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:38.468171 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:38.468175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:38.468176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:38.468265 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:38.468627 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:38.468630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:38.468632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:38.468832 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:38.468902 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:38.469500 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 11:18:38.469626 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:18:38.472631 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:38.454863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.131:50305)
0812 11:18:38.456019 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:38.456072 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:38.457141 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:38.457882 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:38.457891 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:38.458218 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:38.460267 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:38.460287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.461342 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.461345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:38.462649 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:38.462660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:38.463358 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:38.463361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:38.463551 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:38.464185 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:38.464208 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:38.465832 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:38.467734 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:38.467740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:38.467754 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:38.468061 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:38.468358 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:38.468361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:38.468363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:38.468516 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:38.468963 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:38.468969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:38.469250 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 11:18:38.471357 (+ 2107us) client_negotiation.cc:240] Negotiation successful
0812 11:18:38.471679 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":132,"threads_started":1}
I20260812 11:18:38.473867 24114 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:38.474093 24114 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:38.474633 24114 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:38.475828 23167 ts_manager.cc:194] Registered new tserver with Master: 07a8cbe6ded54581ab8235cf22183d46 (127.30.255.131:33549)
I20260812 11:18:38.476538 23167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:50305
I20260812 11:18:38.482775 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:18:38.483567 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 23314
I20260812 11:18:38.523619 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 23572
I20260812 11:18:38.601281 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 23853
I20260812 11:18:38.616130 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 23104
I20260812 11:18:38.630951 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44089
--webserver_interface=127.30.255.190
--webserver_port=37833
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:44089
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:38.760653 24123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:38.760875 24123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:38.760912 24123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:38.764782 24123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:38.764833 24123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:38.764851 24123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:38.764865 24123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:38.764881 24123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:38.770080 24123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:44089
--ranger_config_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=37833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:38.770994 24123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:38.771950 24123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:38.780992 24130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:38.781024 24136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:38.781024 24133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:38.781509 24123 server_base.cc:1034] running on GCE node
I20260812 11:18:38.782045 24123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:38.783041 24123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:38.784209 24123 hybrid_clock.cc:648] HybridClock initialized: now 1786533518784190 us; error 34 us; skew 500 ppm
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:38.786826 24123 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:38.787735 24123 webserver.cc:460] Webserver started at http://127.30.255.190:37833/ using document root <none> and password file <none>
I20260812 11:18:38.788213 24123 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:38.788285 24123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:18:38 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:38.863653 24123 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:18:38.866449 24146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:38.868103 24123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:18:38.868245 24123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "26cc4edc3c1a4df698a38a3a573b7378"
format_stamp: "Formatted at 2026-08-12 11:18:36 on dist-test-slave-gjrz"
server_key: "6956d72d6d3ce2cf1ae901bf31b3b6a9"
server_key_iv: "0cd84d654a8f6a3b4d19d75a987906d4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:38.868988 24123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:38.890862 24123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:38.893934 24123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:38.894119 24123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:38.902758 24123 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:44089
I20260812 11:18:38.902763 24252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:44089 every 8 connection(s)
I20260812 11:18:38.904122 24123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:38.908355 24253 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:18:38.909174 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 24123
I20260812 11:18:38.909677 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:37431
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:44089
--webserver_port=41539
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:38.910276 24253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Bootstrap starting.
I20260812 11:18:38.926906 24253 log.cc:824] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:38.933207 24253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:18:38.933544 24253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Bootstrap complete.
I20260812 11:18:38.937433 24253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:38.937817 24253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26cc4edc3c1a4df698a38a3a573b7378, State: Initialized, Role: FOLLOWER
I20260812 11:18:38.938289 24253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:38.938413 24253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:38.938479 24253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:38.938582 24253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:18:38.940407 24253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:38.940712 24253 leader_election.cc:302] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26cc4edc3c1a4df698a38a3a573b7378; no voters:
I20260812 11:18:38.940994 24253 leader_election.cc:288] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:18:38.941123 24261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:18:38.941318 24261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [term 2 LEADER]: Becoming Leader. State: Replica: 26cc4edc3c1a4df698a38a3a573b7378, State: Running, Role: LEADER
I20260812 11:18:38.941630 24261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } }
I20260812 11:18:38.942201 24253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:38.943042 24266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26cc4edc3c1a4df698a38a3a573b7378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } } }
I20260812 11:18:38.943169 24266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:38.943363 24269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26cc4edc3c1a4df698a38a3a573b7378. Latest consensus state: current_term: 2 leader_uuid: "26cc4edc3c1a4df698a38a3a573b7378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26cc4edc3c1a4df698a38a3a573b7378" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44089 } } }
I20260812 11:18:38.943440 24269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:38.943787 24287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:38.946889 24287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:38.947407 24253 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:18:38.947678 24287 catalog_manager.cc:1259] Loaded cluster ID: 036d0214015d4b0599abe278a89fe1b9
I20260812 11:18:38.947777 24287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:38.950137 24287 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:38.950783 24287 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Loaded TSK: 0
I20260812 11:18:38.954121 24287 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:18:39.052070 24257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:39.052367 24257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:39.052435 24257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:39.058192 24257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:39.058260 24257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:39.058339 24257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:18:39.065588 24257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:37431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=41539
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:39.066741 24257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:39.068005 24257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:39.081008 24335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:39.082150 24334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:39.082202 24337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:39.082818 24257 server_base.cc:1034] running on GCE node
I20260812 11:18:39.083324 24257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:39.084012 24257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:39.101675 24257 hybrid_clock.cc:648] HybridClock initialized: now 1786533519101641 us; error 58 us; skew 500 ppm
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:39.104645 24257 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:18:39.105657 24257 webserver.cc:460] Webserver started at http://127.30.255.129:41539/ using document root <none> and password file <none>
I20260812 11:18:39.106310 24257 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:39.106410 24257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:39.144292 24257 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.000s
I20260812 11:18:39.147693 24361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.149297 24257 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 11:18:39.149432 24257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "91666987d4344ef4bb97a13b5c5aac61"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "6cd87619f7cac4f5aa0e0302af5ff13a"
server_key_iv: "e024c1443806e8ba8e3095b456b265d3"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:39.150178 24257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:39.170733 24257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:39.174476 24257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:39.174674 24257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:39.175383 24257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:39.176421 24257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:39.176496 24257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.176550 24257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:39.176570 24257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.190080 24581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:37431 every 8 connection(s)
I20260812 11:18:39.190263 24257 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:37431
I20260812 11:18:39.191632 24257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:18:39.201393 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 24257
I20260812 11:18:39.201926 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:45729
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:44089
--webserver_port=40615
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:39.213510 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.194937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.129:35547)
0812 11:18:39.196478 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:39.196534 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:39.197559 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:39.199385 (+ 1826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:39.199394 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:39.199749 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:39.200381 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.200402 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.202369 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.202373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.203840 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.203851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.204943 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.204948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.205146 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.205756 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:39.205776 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:39.207528 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:39.209990 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.209996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.210010 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.210337 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.210755 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.210759 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.210761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.210925 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.211690 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:39.211696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:39.211988 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 11:18:39.212166 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 11:18:39.212492 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":157,"threads_started":1}
I20260812 11:18:39.215905 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.196455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:35547 (local address 127.30.255.190:44089)
0812 11:18:39.198052 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:39.198074 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:39.198120 (+ 46us) server_negotiation.cc:407] Connection header received
0812 11:18:39.198414 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:39.198431 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:39.198871 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:39.199228 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:39.200585 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.202207 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.203993 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.204745 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.205557 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.207759 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:39.207765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:39.207781 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:39.207813 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:39.209855 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.210497 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.210503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.210508 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.210647 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.211043 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.211046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.211047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.211556 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:39.214508 (+ 2952us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:39.214656 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 11:18:39.214940 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1438,"thread_start_us":121,"threads_started":1}
I20260812 11:18:39.218815 24582 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:39.219045 24582 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.219640 24582 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
W20260812 11:18:39.222103 24582 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:44089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:18:39.222229 24582 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.222393 24582 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:39.223474 24582 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.223657 24582 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
W20260812 11:18:39.224457 24582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:18:39.366956 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:39.367408 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:39.367537 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:39.373261 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:39.373386 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:39.373519 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:18:39.381057 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:45729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=40615
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:39.382315 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:39.383612 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:39.395872 24608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:39.396315 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:39.396803 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:39.397691 24595 server_base.cc:1034] running on GCE node
I20260812 11:18:39.398138 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:39.398872 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:39.406244 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786533519406162 us; error 88 us; skew 500 ppm
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:39.409801 24595 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:18:39.410880 24595 webserver.cc:460] Webserver started at http://127.30.255.130:40615/ using document root <none> and password file <none>
I20260812 11:18:39.411639 24595 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:39.411764 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:39.448819 24595 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.000s
I20260812 11:18:39.452507 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.454073 24595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:18:39.454191 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "5fe8635066df48cc84d355cf23871772"
format_stamp: "Formatted at 2026-08-12 11:18:37 on dist-test-slave-gjrz"
server_key: "cd778e740a7c7c89fcc6857443f26625"
server_key_iv: "367b04e6ae9d4c92557032a2b7a7a7e4"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:39.454903 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:39.475172 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:39.478876 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:39.479120 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:39.479830 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:39.480934 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:39.481040 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.481133 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:39.481194 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.494380 24595 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:45729
I20260812 11:18:39.495708 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:18:39.496876 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 24595
I20260812 11:18:39.497409 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.255.148:36883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:33549
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:44089
--webserver_port=46175
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:39.496299 24848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:45729 every 8 connection(s)
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:39.516551 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.500479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:38557 (local address 127.30.255.190:44089)
0812 11:18:39.500627 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:39.500631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:39.502887 (+ 2256us) server_negotiation.cc:407] Connection header received
0812 11:18:39.504743 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:39.504746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:39.504803 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:39.504855 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:39.506491 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.507343 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.509019 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.509353 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.509431 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.512403 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:39.512407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:39.512410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:39.512438 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:39.514101 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.514728 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.514732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.514734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.514830 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.515247 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.515251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.515253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.515507 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:39.515581 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:39.516241 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 11:18:39.516369 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:18:39.518261 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.499355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.130:38557)
0812 11:18:39.502714 (+ 3359us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:39.502771 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:39.504476 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:39.505262 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:39.505272 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:39.505656 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:39.506306 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.506325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.507511 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.507514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.508878 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.508889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.509645 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.509648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.509838 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.510471 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:39.510492 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:39.512215 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:39.514230 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.514236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.514251 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.514609 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.514949 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.514952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.514954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.515133 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.515665 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:39.515671 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:39.515968 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 11:18:39.516973 (+ 1005us) client_negotiation.cc:240] Negotiation successful
0812 11:18:39.517290 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3144,"thread_start_us":153,"threads_started":1}
I20260812 11:18:39.519711 24850 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:39.519938 24850 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.520483 24850 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
W20260812 11:18:39.522244 24850 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:44089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:18:39.522364 24850 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.522533 24850 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:39.523399 24850 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.523592 24850 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
W20260812 11:18:39.524417 24850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:18:39.661264 24856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:39.661640 24856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:39.661729 24856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:39.667711 24856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:39.667783 24856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:39.667865 24856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:18:39.673241 24856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.255.148:36883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:33549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=46175
--tserver_master_addrs=127.30.255.190:44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:39.674336 24856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:39.675392 24856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:39.685040 24869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:39.685340 24875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:39.685523 24856 server_base.cc:1034] running on GCE node
W20260812 11:18:39.687405 24870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:39.687856 24856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:39.688647 24856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:39.691956 24856 hybrid_clock.cc:648] HybridClock initialized: now 1786533519691921 us; error 44 us; skew 500 ppm
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:39.695420 24856 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:18:39.696518 24856 webserver.cc:460] Webserver started at http://127.30.255.131:46175/ using document root <none> and password file <none>
I20260812 11:18:39.697302 24856 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:39.697405 24856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:39.719218 24856 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:18:39.722505 24891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.724076 24856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:18:39.724233 24856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "07a8cbe6ded54581ab8235cf22183d46"
format_stamp: "Formatted at 2026-08-12 11:18:38 on dist-test-slave-gjrz"
server_key: "97dca960bd7df25266505f1fded15520"
server_key_iv: "e7ef888a0fc35387e4b446a9a774c2fd"
server_key_version: "kuduclusterkey@0"
I20260812 11:18:39.724941 24856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:39.744066 24856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:39.746917 24856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:39.747145 24856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:39.747782 24856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:39.748610 24856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:39.748657 24856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.748710 24856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:39.748739 24856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:39.759527 24856 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:33549
I20260812 11:18:39.759615 25103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:33549 every 8 connection(s)
I20260812 11:18:39.760584 24856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:18:39.762145 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 24856
I20260812 11:18:39.774434 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.763919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.30.255.190:44089)
0812 11:18:39.764061 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:39.764066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:39.764374 (+ 308us) server_negotiation.cc:407] Connection header received
0812 11:18:39.764607 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:39.764610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:39.764664 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:39.764714 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:39.765733 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.766463 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.767649 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.767973 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.768060 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.769727 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:39.769731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:39.769734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:39.769761 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:39.771386 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.773093 (+ 1707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.773096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.773098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.773188 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.773495 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.773499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.773500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.773694 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:39.773889 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:39.774156 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 11:18:39.774274 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 11:18:39 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:39.780318 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.766208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:47103 (local address 127.30.255.190:44089)
0812 11:18:39.767308 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:39.767312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:39.767326 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:18:39.768211 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:39.768214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:39.768255 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:39.768345 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:39.769806 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.770988 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.772697 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.773299 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.773368 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.777120 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:39.777124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:39.777126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:39.777150 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:39.778357 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.778824 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.778827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.778829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.778958 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:39.779304 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:39.779308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:39.779309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:39.779478 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:39.779571 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:39.780042 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 11:18:39.780156 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1008,"thread_start_us":77,"threads_started":1}
I20260812 11:18:39.781131 25111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:39.765202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44089 (local address 127.30.255.131:47103)
0812 11:18:39.766581 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:39.766638 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:39.767920 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:39.768649 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:39.768660 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:39.769002 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:39.769674 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.769697 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.771145 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.771148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.772558 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:39.772570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:39.773975 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:39.773980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:39.774746 (+ 766us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:39.775285 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:39.775307 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:39.776953 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:39.778464 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.778469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.778483 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.778720 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.779047 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:39.779050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:39.779052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:39.779188 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:39.779587 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:39.779593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:39.779815 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 11:18:39.779963 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 11:18:39.780260 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":125,"threads_started":1}
I20260812 11:18:39.782136 25106 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44089
I20260812 11:18:39.782375 25106 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:39.782905 25106 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:39.784372 24184 ts_manager.cc:194] Registered new tserver with Master: 07a8cbe6ded54581ab8235cf22183d46 (127.30.255.131:33549)
I20260812 11:18:39.785908 24184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:47103
Aug 12 11:18:40 dist-test-slave-gjrz krb5kdc[22229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:40.224650 24582 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:40.224910 24582 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:40.225625 24184 ts_manager.cc:194] Registered new tserver with Master: 91666987d4344ef4bb97a13b5c5aac61 (127.30.255.129:37431)
I20260812 11:18:40.226456 24184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:35547
Aug 12 11:18:40 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:40.524614 24850 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:40.524884 24850 heartbeater.cc:511] Master 127.30.255.190:44089 requested a full tablet report, sending...
I20260812 11:18:40.525574 24184 ts_manager.cc:194] Registered new tserver with Master: 5fe8635066df48cc84d355cf23871772 (127.30.255.130:45729)
I20260812 11:18:40.526387 24184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:38557
I20260812 11:18:40.533908 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:18:40.548877 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.535741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.30.255.190:44089)
0812 11:18:40.536432 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.536435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.536446 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:18:40.536476 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.536478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.536516 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.536594 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:40.537249 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.537772 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.538970 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.539369 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.540214 (+ 845us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.545242 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:40.545246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:40.545249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:40.545279 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:40.547059 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:40.547571 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:40.547575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:40.547577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:40.547675 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:40.547948 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:40.547951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:40.547953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:40.548178 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:40.548312 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.548561 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.548695 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":89,"threads_started":1}
I20260812 11:18:40.553015 24184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:18:40.557861 24184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:18:40.576673 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.569535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.30.255.130:45729)
0812 11:18:40.569895 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.569902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.570654 (+ 752us) server_negotiation.cc:407] Connection header received
0812 11:18:40.570876 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.570882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.571136 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.571259 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:40.571784 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.572993 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.574923 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.575749 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.575844 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.575972 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.576416 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.576519 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":76,"threads_started":1}
I20260812 11:18:40.577310 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.568892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:37431 (local address 127.0.0.1:50284)
0812 11:18:40.570158 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.570204 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:40.570328 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:40.572155 (+ 1827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:40.572158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:40.572176 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:40.572281 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.572287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.573863 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.573866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.575834 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.575846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.576853 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.576856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.576950 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.576973 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:18:40.577016 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 11:18:40.577140 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"thread_start_us":92,"threads_started":1}
I20260812 11:18:40.577644 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.569162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.30.255.129:37431)
0812 11:18:40.571714 (+ 2552us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.571721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.571743 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:18:40.571816 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.571821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.571995 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.572426 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:40.572584 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.573709 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.575983 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.576724 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.577275 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.577395 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.577440 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.577518 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2449,"thread_start_us":113,"threads_started":1}
I20260812 11:18:40.578926 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.568892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:45729 (local address 127.0.0.1:38160)
0812 11:18:40.570558 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.570570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:40.570730 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:40.571491 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:40.571497 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:40.571520 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:40.571644 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.571651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.573153 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.573156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.574492 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.574504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.576092 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.576096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.576205 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.576244 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 11:18:40.577092 (+ 848us) client_negotiation.cc:240] Negotiation successful
0812 11:18:40.577177 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1605,"mutex_wait_us":594,"thread_start_us":95,"threads_started":1}
I20260812 11:18:40.582050 24707 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 (DEFAULT_TABLE table=test-table [id=259bbf64bfa745ba81a5b668fc483774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:40.583195 24707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4d8b67de3a410c8afb9126ed9158b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:40.584728 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.569799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33549 (local address 127.0.0.1:60208)
0812 11:18:40.574713 (+ 4914us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.574726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:40.574807 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:40.579943 (+ 5136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:40.579946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:40.579962 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:40.580037 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.580042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.581403 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.581406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.582516 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.582527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.584373 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.584377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.584461 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.584482 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:18:40.584544 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 11:18:40.584606 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4851,"thread_start_us":82,"threads_started":1}
I20260812 11:18:40.585180 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.570234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.30.255.131:33549)
0812 11:18:40.579404 (+ 9170us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.579411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.579433 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:18:40.579503 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.579508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.579685 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.579812 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:40.580168 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.581273 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.583342 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.584095 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.584203 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.584311 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.584957 (+ 646us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.585051 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9058,"thread_start_us":97,"threads_started":1}
I20260812 11:18:40.587734 24980 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 (DEFAULT_TABLE table=test-table [id=259bbf64bfa745ba81a5b668fc483774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:40.588753 24980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4d8b67de3a410c8afb9126ed9158b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:40.588887 24412 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 (DEFAULT_TABLE table=test-table [id=259bbf64bfa745ba81a5b668fc483774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:40.589862 24412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4d8b67de3a410c8afb9126ed9158b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:40.598639 25168 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Bootstrap starting.
I20260812 11:18:40.600999 25168 tablet_bootstrap.cc:654] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:40.601928 25168 log.cc:824] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:40.603116 25169 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Bootstrap starting.
I20260812 11:18:40.603500 25171 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Bootstrap starting.
I20260812 11:18:40.605660 25171 tablet_bootstrap.cc:654] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:40.606400 25169 tablet_bootstrap.cc:654] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:40.606498 25171 log.cc:824] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:40.607239 25169 log.cc:824] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:40.608668 25168 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: No bootstrap required, opened a new log
I20260812 11:18:40.608850 25168 ts_tablet_manager.cc:1397] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260812 11:18:40.609592 25169 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: No bootstrap required, opened a new log
I20260812 11:18:40.609766 25169 ts_tablet_manager.cc:1397] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 11:18:40.613265 25168 raft_consensus.cc:348] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.613533 25168 raft_consensus.cc:374] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:40.613600 25168 raft_consensus.cc:729] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a8cbe6ded54581ab8235cf22183d46, State: Initialized, Role: FOLLOWER
I20260812 11:18:40.613857 25169 raft_consensus.cc:348] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.614177 25169 raft_consensus.cc:374] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:40.614272 25169 raft_consensus.cc:729] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91666987d4344ef4bb97a13b5c5aac61, State: Initialized, Role: FOLLOWER
I20260812 11:18:40.614799 25169 consensus_queue.cc:260] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.614884 25168 consensus_queue.cc:260] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.615912 25168 ts_tablet_manager.cc:1428] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:18:40.615934 25169 ts_tablet_manager.cc:1428] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 11:18:40.616290 25106 heartbeater.cc:503] Master 127.30.255.190:44089 was elected leader, sending a full tablet report...
I20260812 11:18:40.616340 24582 heartbeater.cc:503] Master 127.30.255.190:44089 was elected leader, sending a full tablet report...
I20260812 11:18:40.620167 25171 tablet_bootstrap.cc:492] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: No bootstrap required, opened a new log
I20260812 11:18:40.620352 25171 ts_tablet_manager.cc:1397] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260812 11:18:40.623833 25171 raft_consensus.cc:348] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.624111 25171 raft_consensus.cc:374] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:40.624210 25171 raft_consensus.cc:729] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe8635066df48cc84d355cf23871772, State: Initialized, Role: FOLLOWER
I20260812 11:18:40.624738 25171 consensus_queue.cc:260] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.625325 24850 heartbeater.cc:503] Master 127.30.255.190:44089 was elected leader, sending a full tablet report...
I20260812 11:18:40.625684 25171 ts_tablet_manager.cc:1428] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
W20260812 11:18:40.695322 24584 tablet.cc:2369] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:40.731182 25183 raft_consensus.cc:482] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:40.731496 25183 raft_consensus.cc:504] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.732954 25183 leader_election.cc:288] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a8cbe6ded54581ab8235cf22183d46 (127.30.255.131:33549), 91666987d4344ef4bb97a13b5c5aac61 (127.30.255.129:37431)
I20260812 11:18:40.738962 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.733818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:56271 (local address 127.30.255.129:37431)
0812 11:18:40.733986 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.733991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.734214 (+ 223us) server_negotiation.cc:407] Connection header received
0812 11:18:40.734450 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.734453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.734510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.734558 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:40.735082 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.735923 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.737601 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.738234 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.738329 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.738383 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.738758 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.738837 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 11:18:40.738979 25187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.733243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:37431 (local address 127.30.255.130:56271)
0812 11:18:40.734113 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.734127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:40.734311 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:40.734718 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:40.734720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:40.734740 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:40.734886 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.734904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.736079 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.736083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.737448 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.737460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.738486 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.738490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.738612 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.738635 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:18:40.738720 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 11:18:40.738790 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":13,"thread_start_us":128,"threads_started":1}
I20260812 11:18:40.739879 24495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4d8b67de3a410c8afb9126ed9158b5" candidate_uuid: "5fe8635066df48cc84d355cf23871772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91666987d4344ef4bb97a13b5c5aac61" is_pre_election: true
I20260812 11:18:40.740219 24495 raft_consensus.cc:2454] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe8635066df48cc84d355cf23871772 in term 0.
I20260812 11:18:40.740639 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.733770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:60339 (local address 127.30.255.131:33549)
0812 11:18:40.733921 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.733926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.736037 (+ 2111us) server_negotiation.cc:407] Connection header received
0812 11:18:40.736251 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.736255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.736307 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.736357 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:40.736806 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.737564 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.739392 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.739960 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.740052 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.740098 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.740455 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.740527 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:18:40.740733 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.733351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:33549 (local address 127.30.255.130:60339)
0812 11:18:40.735941 (+ 2590us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.735956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:40.736116 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:40.736502 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:40.736505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:40.736535 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:40.736665 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.736672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.737729 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.737732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.738990 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:40.739001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.740198 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.740201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:40.740301 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.740323 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:18:40.740434 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 11:18:40.740608 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"thread_start_us":120,"threads_started":1}
I20260812 11:18:40.740748 24628 leader_election.cc:302] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fe8635066df48cc84d355cf23871772, 91666987d4344ef4bb97a13b5c5aac61; no voters:
I20260812 11:18:40.741048 25183 raft_consensus.cc:2781] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:18:40.741137 25183 raft_consensus.cc:482] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:18:40.741178 25183 raft_consensus.cc:3037] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:40.741900 25013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4d8b67de3a410c8afb9126ed9158b5" candidate_uuid: "5fe8635066df48cc84d355cf23871772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a8cbe6ded54581ab8235cf22183d46" is_pre_election: true
I20260812 11:18:40.742164 25013 raft_consensus.cc:2454] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe8635066df48cc84d355cf23871772 in term 0.
I20260812 11:18:40.742501 25183 raft_consensus.cc:504] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
I20260812 11:18:40.742859 25183 leader_election.cc:288] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [CANDIDATE]: Term 1 election: Requested vote from peers 07a8cbe6ded54581ab8235cf22183d46 (127.30.255.131:33549), 91666987d4344ef4bb97a13b5c5aac61 (127.30.255.129:37431)
I20260812 11:18:40.743435 24495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4d8b67de3a410c8afb9126ed9158b5" candidate_uuid: "5fe8635066df48cc84d355cf23871772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91666987d4344ef4bb97a13b5c5aac61"
I20260812 11:18:40.743553 24495 raft_consensus.cc:3037] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:40.743603 25013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4d8b67de3a410c8afb9126ed9158b5" candidate_uuid: "5fe8635066df48cc84d355cf23871772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a8cbe6ded54581ab8235cf22183d46"
I20260812 11:18:40.743713 25013 raft_consensus.cc:3037] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:40.744800 24495 raft_consensus.cc:2454] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe8635066df48cc84d355cf23871772 in term 1.
I20260812 11:18:40.744980 25013 raft_consensus.cc:2454] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe8635066df48cc84d355cf23871772 in term 1.
I20260812 11:18:40.745115 24628 leader_election.cc:302] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fe8635066df48cc84d355cf23871772, 91666987d4344ef4bb97a13b5c5aac61; no voters:
Aug 12 11:18:40 dist-test-slave-gjrz krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for HTTP/127.30.255.148@KRBTEST.COM
I20260812 11:18:40.745985 25183 raft_consensus.cc:2781] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:40.746199 25183 raft_consensus.cc:686] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe8635066df48cc84d355cf23871772, State: Running, Role: LEADER
I20260812 11:18:40.746543 25183 consensus_queue.cc:237] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } }
W20260812 11:18:40.747851 24852 tablet.cc:2369] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:40.750133 24184 catalog_manager.cc:5495] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe8635066df48cc84d355cf23871772 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "5fe8635066df48cc84d355cf23871772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fe8635066df48cc84d355cf23871772" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 45729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 } health_report { overall_health: UNKNOWN } } }
W20260812 11:18:40.762610 25107 tablet.cc:2369] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:40.774936 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.770236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.30.255.130:45729)
0812 11:18:40.770565 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:40.770569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:40.770580 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:18:40.770638 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:40.770640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:40.770687 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:40.770774 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:18:40.771069 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.771906 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.773155 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:40.773377 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:40.773464 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:40.773803 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:18:40.774302 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:18:40.774410 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:40.774728 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 11:18:40.774787 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260812 11:18:40.781085 25013 raft_consensus.cc:1261] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 1 FOLLOWER]: Refusing update from remote peer 5fe8635066df48cc84d355cf23871772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:40.781088 24495 raft_consensus.cc:1261] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 1 FOLLOWER]: Refusing update from remote peer 5fe8635066df48cc84d355cf23871772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:40.782014 25183 consensus_queue.cc:1035] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91666987d4344ef4bb97a13b5c5aac61" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:40.782266 25192 consensus_queue.cc:1035] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a8cbe6ded54581ab8235cf22183d46" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 33549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:40.793835 25201 mvcc.cc:205] Tried to move back new op lower bound from 7317641301111386112 to 7317641300981399552. Current Snapshot: MvccSnapshot[applied={T|T < 7317641301111386112}]
I20260812 11:18:40.794425 25199 mvcc.cc:205] Tried to move back new op lower bound from 7317641301111386112 to 7317641300981399552. Current Snapshot: MvccSnapshot[applied={T|T < 7317641301111386112}]
I20260812 11:18:40.803937 24184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:40.804102 24184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:40.805626 25208 mvcc.cc:205] Tried to move back new op lower bound from 7317641301111386112 to 7317641300981399552. Current Snapshot: MvccSnapshot[applied={T|T < 7317641301111386112}]
I20260812 11:18:40.807637 24184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26cc4edc3c1a4df698a38a3a573b7378: Sending DeleteTablet for 3 replicas of tablet 4a4d8b67de3a410c8afb9126ed9158b5
I20260812 11:18:40.808498 24412 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:40 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50284
I20260812 11:18:40.808485 24980 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:40 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:60208
I20260812 11:18:40.808961 25226 tablet_replica.cc:330] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: stopping tablet replica
I20260812 11:18:40.809110 24707 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4d8b67de3a410c8afb9126ed9158b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:40 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:38160
I20260812 11:18:40.809186 25229 tablet_replica.cc:330] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61: stopping tablet replica
I20260812 11:18:40.809491 25229 raft_consensus.cc:2229] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:40.809756 25229 raft_consensus.cc:2258] T 4a4d8b67de3a410c8afb9126ed9158b5 P 91666987d4344ef4bb97a13b5c5aac61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:40.810211 25226 raft_consensus.cc:2229] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:40.810570 25226 raft_consensus.cc:2258] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:40.812238 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 24257
I20260812 11:18:40.813624 25226 ts_tablet_manager.cc:1905] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:40.816466 25226 ts_tablet_manager.cc:1918] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:40.822196 25232 tablet_replica.cc:330] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: stopping tablet replica
I20260812 11:18:40.822556 25232 raft_consensus.cc:2229] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:18:40.823009 25232 raft_consensus.cc:2258] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:40.823174 25226 log.cc:1196] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/4a4d8b67de3a410c8afb9126ed9158b5
I20260812 11:18:40.823619 25226 ts_tablet_manager.cc:1939] T 4a4d8b67de3a410c8afb9126ed9158b5 P 07a8cbe6ded54581ab8235cf22183d46: Deleting consensus metadata
I20260812 11:18:40.827150 24151 catalog_manager.cc:4841] TS 07a8cbe6ded54581ab8235cf22183d46 (127.30.255.131:33549): tablet 4a4d8b67de3a410c8afb9126ed9158b5 (table test-table [id=259bbf64bfa745ba81a5b668fc483774]) successfully deleted
I20260812 11:18:40.828178 25232 ts_tablet_manager.cc:1905] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:40.834205 25232 ts_tablet_manager.cc:1918] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:40.834367 25232 log.cc:1196] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/4a4d8b67de3a410c8afb9126ed9158b5
I20260812 11:18:40.834764 25232 ts_tablet_manager.cc:1939] T 4a4d8b67de3a410c8afb9126ed9158b5 P 5fe8635066df48cc84d355cf23871772: Deleting consensus metadata
I20260812 11:18:40.835872 24151 catalog_manager.cc:4841] TS 5fe8635066df48cc84d355cf23871772 (127.30.255.130:45729): tablet 4a4d8b67de3a410c8afb9126ed9158b5 (table test-table [id=259bbf64bfa745ba81a5b668fc483774]) successfully deleted
W20260812 11:18:40.846482 24153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:37431 (error 108)
I20260812 11:18:40.847355 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 24595
I20260812 11:18:40.847558 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:40.847241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:37431 (local address 127.0.0.1:50300)
0812 11:18:40.847395 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:40.847469 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:37431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 11:18:40.847724 24153 catalog_manager.cc:4584] TS 91666987d4344ef4bb97a13b5c5aac61 (127.30.255.129:37431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a4d8b67de3a410c8afb9126ed9158b5: Network error: Client connection negotiation failed: client connection to 127.30.255.129:37431: connect: Connection refused (error 111)
I20260812 11:18:40.858610 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 24856
I20260812 11:18:40.869971 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 24123
I20260812 11:18:40.885661 31742 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 11:18:41.233875 31742 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 11:18:41.233992 31742 mini_ranger.cc:66] Stopping Ranger...
I20260812 11:18:41.588467 31742 mini_ranger.cc:68] Stopped Ranger
2026-08-12 11:18:41.588 UTC [22282] LOG: received smart shutdown request
2026-08-12 11:18:41.592 UTC [22282] LOG: background worker "logical replication launcher" (PID 22297) exited with exit code 1
2026-08-12 11:18:41.593 UTC [22289] LOG: shutting down
2026-08-12 11:18:41.719 UTC [22282] LOG: database system is shut down
2026-08-12T11:18:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47180 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:18:41 dist-test-slave-gjrz krb5kdc[25250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:18:41 dist-test-slave-gjrz krb5kdc[25250](info): set up 2 sockets
Aug 12 11:18:41 dist-test-slave-gjrz krb5kdc[25250](info): commencing operation
krb5kdc: starting...
W20260812 11:18:43.013736 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:18:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:18:43.127903 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43303
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:43.235824 25269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:43.236064 25269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:43.236117 25269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:43.239981 25269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:43.240031 25269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:43.240058 25269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:43.240079 25269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:43.240108 25269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:43.245167 25269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:43.246134 25269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:43.247088 25269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:43.254976 25282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:43.255134 25287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:43.255014 25269 server_base.cc:1034] running on GCE node
W20260812 11:18:43.255021 25276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:43.255741 25269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:43.256742 25269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:43.257901 25269 hybrid_clock.cc:648] HybridClock initialized: now 1786533523257882 us; error 29 us; skew 500 ppm
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:43.260411 25269 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:43.261227 25269 webserver.cc:460] Webserver started at http://127.30.255.190:43425/ using document root <none> and password file <none>
I20260812 11:18:43.261737 25269 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:43.261801 25269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:43.262014 25269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:43.263962 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "7335a81b5ee54251b1fda900e91ceb88"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "98b1efe4b5154c0ea8b09f0217e332dc"
server_key_iv: "75462779eba075040956f9237610684a"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.264472 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "7335a81b5ee54251b1fda900e91ceb88"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "98b1efe4b5154c0ea8b09f0217e332dc"
server_key_iv: "75462779eba075040956f9237610684a"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.267899 25269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:18:43.270035 25298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:43.271130 25269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:18:43.271238 25269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "7335a81b5ee54251b1fda900e91ceb88"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "98b1efe4b5154c0ea8b09f0217e332dc"
server_key_iv: "75462779eba075040956f9237610684a"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.271353 25269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:43.293857 25269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:43.296778 25269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:43.296962 25269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:43.303486 25269 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:43303
I20260812 11:18:43.303735 25407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:43303 every 8 connection(s)
I20260812 11:18:43.304365 25269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:43.307102 25408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:43.310765 25408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.313097 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 25269
I20260812 11:18:43.313233 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:18:43.313524 31742 external_mini_cluster.cc:1428] Setting key b29bc5ce9f3f6624829ab5283dc918f6
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.189@KRBTEST.COM" created.
I20260812 11:18:43.324721 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.312882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.30.255.190:43303)
0812 11:18:43.313267 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.313284 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.313552 (+ 268us) server_negotiation.cc:407] Connection header received
0812 11:18:43.314567 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.314571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.314741 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.314904 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.315756 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.316597 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.317536 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.317884 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.318026 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.320254 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.320259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.320268 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.320288 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.321776 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.322281 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.322286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.322287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.322386 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.322673 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.322675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.322677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.323017 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.323134 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.323592 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.323751 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":248,"thread_start_us":99,"threads_started":1}
I20260812 11:18:43.324721 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.312883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43303 (local address 127.0.0.1:57910)
0812 11:18:43.313415 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.313454 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.314277 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.314917 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.314922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.315371 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.315613 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.315626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.316713 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.316717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.317423 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.317433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.318027 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.318031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.318183 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.318721 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.318736 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.320123 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.321873 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.321895 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.321904 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.322148 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.322476 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.322479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.322481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.322588 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.323127 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.323133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.323284 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.323452 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.323814 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":101,"thread_start_us":149,"threads_started":1}
I20260812 11:18:43.326870 25408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.327535 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.327299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:33984)
0812 11:18:43.327392 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.327472 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 11:18:43.327746 25304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
I20260812 11:18:43.328547 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.328332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:33988)
0812 11:18:43.328447 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.328491 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 11:18:43.328670 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34013: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
I20260812 11:18:43.338061 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:34013
--webserver_interface=127.30.255.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:43.377883 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } attempt: 1
I20260812 11:18:43.378702 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.378391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:33994)
0812 11:18:43.378557 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.378619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 11:18:43.379535 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.379319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34006)
0812 11:18:43.379418 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.379465 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 11:18:43.379654 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34013: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
W20260812 11:18:43.445948 25419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:43.446177 25419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:43.446213 25419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:18:43.447796 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } attempt: 2
I20260812 11:18:43.448606 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.448307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34020)
0812 11:18:43.448453 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.448526 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:18:43.449276 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.449045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34026)
0812 11:18:43.449170 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.449219 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 11:18:43.449390 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34013: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34013: connect: Connection refused (error 111)
W20260812 11:18:43.450362 25419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:43.450408 25419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:43.450426 25419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:43.450441 25419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:43.450454 25419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:43.455521 25419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:34013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.255.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:43.456423 25419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:43.457309 25419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:43.465201 25429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:43.465216 25419 server_base.cc:1034] running on GCE node
W20260812 11:18:43.465230 25425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:43.465512 25434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:43.466017 25419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:43.467031 25419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:43.468187 25419 hybrid_clock.cc:648] HybridClock initialized: now 1786533523468149 us; error 48 us; skew 500 ppm
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:43.470726 25419 init.cc:377] Logged in from keytab as kudu/127.30.255.189@KRBTEST.COM (short username kudu)
I20260812 11:18:43.471593 25419 webserver.cc:460] Webserver started at http://127.30.255.189:33419/ using document root <none> and password file <none>
I20260812 11:18:43.472097 25419 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:43.472162 25419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:43.472358 25419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:43.474409 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data/instance:
uuid: "7d50ce5e6a1b4066b386b353325051be"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fda74e9e0054966633adc4d65d264d4a"
server_key_iv: "ab00ce0906dc54e9e4da8f120ed28dfd"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.474928 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal/instance:
uuid: "7d50ce5e6a1b4066b386b353325051be"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fda74e9e0054966633adc4d65d264d4a"
server_key_iv: "ab00ce0906dc54e9e4da8f120ed28dfd"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.478387 25419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:18:43.480618 25444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:43.481756 25419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:18:43.481870 25419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "7d50ce5e6a1b4066b386b353325051be"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fda74e9e0054966633adc4d65d264d4a"
server_key_iv: "ab00ce0906dc54e9e4da8f120ed28dfd"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.481976 25419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:43.497774 25419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:43.501151 25419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:43.501345 25419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:43.509039 25419 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.189:34013
I20260812 11:18:43.509265 25536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.189:34013 every 8 connection(s)
I20260812 11:18:43.509933 25419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
I20260812 11:18:43.512576 25537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:43.513870 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 25419
I20260812 11:18:43.513999 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-1/wal/instance
I20260812 11:18:43.514283 31742 external_mini_cluster.cc:1428] Setting key d78d64b42a7ebc4c1987eefc770c6760
I20260812 11:18:43.516163 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.255.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.188@KRBTEST.COM" created.
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:43.529081 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.517949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.30.255.190:43303)
0812 11:18:43.518092 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.518096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.518833 (+ 737us) server_negotiation.cc:407] Connection header received
0812 11:18:43.519803 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.519807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.519871 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.519941 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.521018 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.521540 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.522682 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.522953 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.523043 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.525425 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.525429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.525431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.525455 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.526976 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.527554 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.527558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.527559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.527639 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.527989 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.527991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.527992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.528161 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.528234 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.528850 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.528959 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:18:43.530072 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.518236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43303 (local address 127.0.0.1:57926)
0812 11:18:43.518685 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.518736 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.519605 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.520113 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.520120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.520430 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.520875 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.520888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.521659 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.521662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.522566 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.522576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.523097 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.523101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.523253 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.523742 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.523757 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.525271 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.527104 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.527108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.527119 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.527456 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.527728 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.527731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.527733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.527900 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.528259 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.528266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.528576 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.528777 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.529105 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260812 11:18:43.531528 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
I20260812 11:18:43.539834 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.532010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34038)
0812 11:18:43.532162 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.532173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.532231 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.532742 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.532745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.532909 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.532985 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.532990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.533985 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.533988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.534555 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.534563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.535133 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.535137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.535221 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.535575 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.535586 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.536722 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.538355 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.538358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.538360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.538568 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.538830 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.538833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.538835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.538916 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.539375 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.539378 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.539463 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.539555 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.539689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 11:18:43.539958 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.532123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.30.255.189:34013)
0812 11:18:43.532435 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.532444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.532462 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:18:43.532493 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.532497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.532629 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.532743 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.533085 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.533845 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.534657 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.535004 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.535095 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.536849 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.536859 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.536873 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.536898 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.538248 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.538666 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.538671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.538672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.538751 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.538995 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.538998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.539000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.539237 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.539415 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.539665 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.539789 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":137,"threads_started":1}
I20260812 11:18:43.540275 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:37485
--webserver_interface=127.30.255.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:43.540541 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } attempt: 3
I20260812 11:18:43.541905 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.542608 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.542365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53416)
0812 11:18:43.542462 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.542540 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 11:18:43.542850 25454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
I20260812 11:18:43.543751 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.543531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53428)
0812 11:18:43.543638 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.543690 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 11:18:43.543968 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:18:43.549219 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.540930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34052)
0812 11:18:43.541067 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.541081 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.541160 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.541991 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.541994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.542156 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.542277 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.542283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.543114 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.543117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.544320 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.544331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.544778 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.544782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.544894 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.545284 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.545294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.546386 (+ 1092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.547873 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.547884 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.547886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.548068 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.548317 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.548321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.548322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.548403 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.548734 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.548738 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.548841 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.548934 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.549064 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:18:43.549218 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.541650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.30.255.189:34013)
0812 11:18:43.541755 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.541759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.541772 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:18:43.541805 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.541808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.541847 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.541931 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.542391 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.543012 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.544431 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.544678 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.544764 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.546510 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.546513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.546516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.546540 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.547761 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.548159 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.548162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.548164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.548234 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.548482 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.548485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.548487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.548644 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.548726 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.548997 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.549092 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:18:43.550159 25408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.550782 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.550572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53442)
0812 11:18:43.550680 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.550724 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 11:18:43.551435 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.551193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53448)
0812 11:18:43.551338 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.551385 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 11:18:43.551545 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
I20260812 11:18:43.582695 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 1
I20260812 11:18:43.583442 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.583146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53454)
0812 11:18:43.583313 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.583371 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 11:18:43.584093 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.583874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53458)
0812 11:18:43.583987 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.584038 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 11:18:43.584208 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
I20260812 11:18:43.593189 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 1
I20260812 11:18:43.593837 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.593575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53470)
0812 11:18:43.593723 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.593773 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 11:18:43.594544 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.594339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53478)
0812 11:18:43.594440 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.594483 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 11:18:43.594656 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
W20260812 11:18:43.649734 25550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:43.649969 25550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:43.650024 25550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:43.653884 25550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:43.653934 25550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:43.653960 25550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:43.653981 25550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:43.654006 25550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:43.659008 25550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43303,127.30.255.189:34013,127.30.255.188:37485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:37485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.255.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:43.659310 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 2
I20260812 11:18:43.660023 25550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:43.660058 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.659767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53488)
0812 11:18:43.659900 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.659959 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:18:43.660719 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.660513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53494)
0812 11:18:43.660618 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.660662 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 11:18:43.660835 25408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
I20260812 11:18:43.661019 25550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:18:43.662792 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 2
I20260812 11:18:43.663388 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.663134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53508)
0812 11:18:43.663251 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.663325 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 11:18:43.664002 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.663806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53510)
0812 11:18:43.663901 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.663947 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 11:18:43.664110 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:37485: Network error: Client connection negotiation failed: client connection to 127.30.255.188:37485: connect: Connection refused (error 111)
W20260812 11:18:43.669025 25568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:43.669060 25572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:43.669037 25569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:43.669245 25550 server_base.cc:1034] running on GCE node
I20260812 11:18:43.669847 25550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:43.670861 25550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:43.672014 25550 hybrid_clock.cc:648] HybridClock initialized: now 1786533523672009 us; error 40 us; skew 500 ppm
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:43.674606 25550 init.cc:377] Logged in from keytab as kudu/127.30.255.188@KRBTEST.COM (short username kudu)
I20260812 11:18:43.675499 25550 webserver.cc:460] Webserver started at http://127.30.255.188:40807/ using document root <none> and password file <none>
I20260812 11:18:43.676062 25550 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:43.676131 25550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:43.676323 25550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:43.678223 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data/instance:
uuid: "3837ef2345c3439ebb5bcc00c82b7c39"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fe48a0f2b6e60c856d824037b36d45bf"
server_key_iv: "8ed218b5b5eb37a00cd08f15112d9211"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.678764 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal/instance:
uuid: "3837ef2345c3439ebb5bcc00c82b7c39"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fe48a0f2b6e60c856d824037b36d45bf"
server_key_iv: "8ed218b5b5eb37a00cd08f15112d9211"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.682210 25550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:18:43.684463 25583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:43.685556 25550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:18:43.685673 25550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "3837ef2345c3439ebb5bcc00c82b7c39"
format_stamp: "Formatted at 2026-08-12 11:18:43 on dist-test-slave-gjrz"
server_key: "fe48a0f2b6e60c856d824037b36d45bf"
server_key_iv: "8ed218b5b5eb37a00cd08f15112d9211"
server_key_version: "encryptionkey@0"
I20260812 11:18:43.685760 25550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:43.697726 25550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:43.700528 25550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:43.700695 25550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:43.707070 25550 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.188:37485
I20260812 11:18:43.707353 25692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.188:37485 every 8 connection(s)
I20260812 11:18:43.707974 25550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
I20260812 11:18:43.710577 25693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:43.714056 25693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.716621 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 25550
I20260812 11:18:43.716764 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786533365564394-31742-0/minicluster-data/master-2/wal/instance
I20260812 11:18:43.717022 31742 external_mini_cluster.cc:1428] Setting key d4628ad89ccc26af47a86a1d99476f95
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:18:43.727099 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.715748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.30.255.190:43303)
0812 11:18:43.715872 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.715876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.716614 (+ 738us) server_negotiation.cc:407] Connection header received
0812 11:18:43.717518 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.717522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.717578 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.717658 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.719026 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.719786 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.720940 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.721156 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.721252 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.723648 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.723652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.723654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.723679 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.725038 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.725545 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.725549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.725551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.725644 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.725968 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.725971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.725972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.726195 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.726297 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.726812 (+ 515us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.726934 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:18:43.728056 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.717581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.30.255.188:37485)
0812 11:18:43.717979 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.717987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.718009 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:18:43.718122 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.718126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.718271 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.718397 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.719202 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.720257 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.721238 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.721669 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.721837 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.724053 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.724061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.724073 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.724097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.725576 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.725955 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.725960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.725962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.726040 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.726296 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.726299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.726300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.726590 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.726772 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.726947 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.727112 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":319,"thread_start_us":159,"threads_started":1}
I20260812 11:18:43.728056 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.716069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:43303 (local address 127.0.0.1:57934)
0812 11:18:43.716480 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.716518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.717328 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.717842 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.717851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.718250 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.718869 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.718889 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.719891 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.719894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.720821 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.720834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.721288 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.721291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.721430 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.721979 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.721999 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.723506 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.725141 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.725150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.725157 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.725458 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.725742 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.725745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.725746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.725875 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.726300 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.726306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.726598 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.726763 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.727066 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":28,"spinlock_wait_cycles":1408,"thread_start_us":97,"threads_started":1}
I20260812 11:18:43.729224 25693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:43.737352 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.729647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34013 (local address 127.0.0.1:34056)
0812 11:18:43.729803 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.729816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.729883 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.730153 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.730156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.730303 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.730418 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.730425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.731401 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.731404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.732103 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.732111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.732633 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.732635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.732701 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.733054 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.733067 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.734254 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.736008 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.736012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.736013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.736227 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.736473 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.736476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.736478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.736565 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.736901 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.736904 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.736988 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.737081 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.737205 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 11:18:43.737375 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.729752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.30.255.189:34013)
0812 11:18:43.729867 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.729871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.729884 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:18:43.729958 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.729961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.730028 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.730103 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.730535 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.731309 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.732203 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.732542 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.732639 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.734391 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.734395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.734397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.734424 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.735909 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.736314 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.736317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.736318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.736389 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.736653 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.736655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.736656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.736804 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.736885 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.737147 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.737243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:18:43.738049 25693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:18:43.738199 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.730230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.30.255.190:43303)
0812 11:18:43.730328 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.730331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.730343 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:18:43.730416 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.730419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.730457 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.730528 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.731187 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.731872 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.732807 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.733069 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.733184 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.735412 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.735416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.735417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.735438 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.736737 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.737076 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.737080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.737089 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.737159 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.737387 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.737390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.737392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.737597 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.737802 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.737947 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.738052 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:18:43.745395 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.738461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53534)
0812 11:18:43.738551 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.738564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.738634 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.739034 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.739037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.739175 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.739246 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.739252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.740044 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.740047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.740826 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.740836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.741256 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.741260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.741341 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.741707 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.741719 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.742729 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.744184 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.744187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.744189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.744354 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.744575 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.744577 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.744579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.744648 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.744950 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.744953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.745029 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.745115 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.745237 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 11:18:43.745415 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.738781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.30.255.188:37485)
0812 11:18:43.738878 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.738882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.738893 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:18:43.738916 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.738918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.738947 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.739017 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.739368 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.739935 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.740922 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.741156 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.741226 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.742843 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.742846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.742849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.742869 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.744093 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.744433 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.744436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.744437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.744500 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.744721 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.744724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.744725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.744866 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.744945 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.745169 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.745289 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":36}
I20260812 11:18:43.747255 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.739205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.30.255.189:34013)
0812 11:18:43.739324 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.739328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.739360 (+ 32us) server_negotiation.cc:407] Connection header received
0812 11:18:43.739476 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.739479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.739518 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.739597 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.740097 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.740814 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.741682 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.741914 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.741997 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.744228 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.744232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.744235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.744259 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.745784 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.746210 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.746213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.746215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.746289 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.746532 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.746536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.746537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.746698 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.746903 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.747027 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.747126 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:18:43.748624 25693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: Bootstrap starting.
I20260812 11:18:43.750417 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:18:43.751147 25693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:43.751933 25693 log.cc:824] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:43.754179 25693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: No bootstrap required, opened a new log
I20260812 11:18:43.755255 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 3
I20260812 11:18:43.757858 25693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.758126 25693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:43.758185 25693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3837ef2345c3439ebb5bcc00c82b7c39, State: Initialized, Role: FOLLOWER
I20260812 11:18:43.758668 25693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.759182 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.759419 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.759972 25408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } attempt: 3
I20260812 11:18:43.760161 25693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:43.760742 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.752447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.30.255.188:37485)
0812 11:18:43.752578 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.752592 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.752921 (+ 329us) server_negotiation.cc:407] Connection header received
0812 11:18:43.753077 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.753090 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.753131 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.753174 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.754044 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.754721 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.755600 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.755890 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.755974 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.757343 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.757346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.757349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.757372 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.758638 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.759136 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.759140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.759142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.759230 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.759618 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.759621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.759623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.759808 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.759929 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.760496 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.760609 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 11:18:43.763334 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.753026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.30.255.189:34013)
0812 11:18:43.753121 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.753124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.753136 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:18:43.753167 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.753170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.753203 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.753265 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.753863 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.754563 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.756477 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.756788 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.756901 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.759594 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.759598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.759600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.759623 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.761157 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.761974 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.761977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.761978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.762063 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.762392 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.762396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.762397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.762599 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.762691 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.763044 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.763164 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
W20260812 11:18:43.768369 25744 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:18:43.768551 25744 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:18:43.763630 25550 master_runner.cc:186] Error getting master registration for 127.30.255.190:43303: OK, instance_id { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" instance_seqno: 1786533523301683 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:18:43.768846 25550 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:18:43.768930 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.755627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53552)
0812 11:18:43.755746 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.755760 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.755824 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.761060 (+ 5236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.761063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.761385 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.761468 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.761475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.762347 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.762350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.763046 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.763058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.763614 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.763618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.763699 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.764088 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.764102 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.765506 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.767403 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.767407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.767410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.767652 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.767973 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.767977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.767979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.768080 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.768488 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.768491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.768577 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.768668 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.768792 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:18:43.769435 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.755830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.30.255.188:37485)
0812 11:18:43.760857 (+ 5027us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.760861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.760873 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:18:43.760908 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.760911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.760947 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.761024 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.761578 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.762226 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.763187 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.763494 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.763892 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.765648 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.765651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.765654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.765676 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.767228 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.767779 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.767782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.767784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.767867 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.768189 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.768192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.768193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.768376 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.769188 (+ 812us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.769257 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.769335 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4976,"thread_start_us":94,"threads_started":1}
I20260812 11:18:43.770318 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.752393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.30.255.190:43303)
0812 11:18:43.752504 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.752507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.762441 (+ 9934us) server_negotiation.cc:407] Connection header received
0812 11:18:43.762616 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.762620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.762663 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.762707 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.763451 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.764120 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.765064 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.765375 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.765469 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.766941 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.766945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.766947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.766971 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.768482 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.768955 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.768959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.768960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.769044 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.769347 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.769351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.769352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.769561 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.769656 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.770018 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.770167 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:18:43.772315 25537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: Bootstrap starting.
Aug 12 11:18:43 dist-test-slave-gjrz krb5kdc[25250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:18:43.776605 25537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:43.777920 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.761133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:37485 (local address 127.0.0.1:53560)
0812 11:18:43.761312 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:43.761324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:43.761395 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:43.769965 (+ 8570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:43.769968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:43.770169 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:43.770261 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.770268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.771227 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.771230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.771963 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:43.771974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.772528 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.772532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:43.772604 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.773041 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:43.773054 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:43.774426 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:43.776369 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.776372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.776374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.776604 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.776930 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:43.776933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:43.776934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:43.777046 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:43.777470 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:43.777473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:43.777566 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 11:18:43.777661 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 11:18:43.777776 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 11:18:43.778151 25537 log.cc:824] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:43.778687 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:43.760639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.30.255.188:37485)
0812 11:18:43.769758 (+ 9119us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:43.769762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:43.769774 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:18:43.769803 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:43.769806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:43.769842 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:43.770364 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:43.770447 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.771097 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.772119 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:43.772399 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:43.772795 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:43.774719 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:43.774725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:43.774727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:43.774750 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:43.776241 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.776732 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.776735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.776736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.776818 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:43.777160 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:43.777163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:43.777164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:43.777355 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:43.778371 (+ 1016us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:43.778443 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 11:18:43.778547 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9070,"thread_start_us":112,"threads_started":1}
I20260812 11:18:43.780135 25537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: No bootstrap required, opened a new log
I20260812 11:18:43.781545 25408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: Bootstrap starting.
I20260812 11:18:43.783037 25537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.783212 25537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:43.783329 25537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d50ce5e6a1b4066b386b353325051be, State: Initialized, Role: FOLLOWER
I20260812 11:18:43.783802 25537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.784009 25408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:43.784796 25408 log.cc:824] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:43.785166 25537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:43.785535 25747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.785717 25747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.786775 25408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: No bootstrap required, opened a new log
W20260812 11:18:43.789981 25773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:18:43.790050 25773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:18:43.790468 25408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.790729 25408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:43.790815 25408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7335a81b5ee54251b1fda900e91ceb88, State: Initialized, Role: FOLLOWER
I20260812 11:18:43.791282 25408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.792491 25408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:43.792479 25419 master_runner.cc:186] Error getting master registration for 127.30.255.190:43303: OK, instance_id { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" instance_seqno: 1786533523301683 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:18:43.795661 25419 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 11:18:43.797201 25791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:18:43.797312 25791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 11:18:43.798375 25309 tablet.cc:2369] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:43.801040 25269 master_runner.cc:263] No leader master found from master 7335a81b5ee54251b1fda900e91ceb88
I20260812 11:18:43.801103 25269 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:18:43.803462 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.803651 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.861961 25747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:43.862176 25747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.863929 25352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d50ce5e6a1b4066b386b353325051be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7335a81b5ee54251b1fda900e91ceb88" is_pre_election: true
I20260812 11:18:43.864241 25352 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d50ce5e6a1b4066b386b353325051be in term 0.
I20260812 11:18:43.864759 25453 leader_election.cc:302] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7335a81b5ee54251b1fda900e91ceb88, 7d50ce5e6a1b4066b386b353325051be; no voters:
I20260812 11:18:43.866156 25646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d50ce5e6a1b4066b386b353325051be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" is_pre_election: true
I20260812 11:18:43.866410 25646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d50ce5e6a1b4066b386b353325051be in term 0.
I20260812 11:18:43.863220 25747 leader_election.cc:288] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7335a81b5ee54251b1fda900e91ceb88 (127.30.255.190:43303), 3837ef2345c3439ebb5bcc00c82b7c39 (127.30.255.188:37485)
I20260812 11:18:43.867038 25747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:18:43.867110 25747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:18:43.867146 25747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:43.868352 25747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.868666 25747 leader_election.cc:288] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [CANDIDATE]: Term 1 election: Requested vote from peers 7335a81b5ee54251b1fda900e91ceb88 (127.30.255.190:43303), 3837ef2345c3439ebb5bcc00c82b7c39 (127.30.255.188:37485)
I20260812 11:18:43.869166 25646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d50ce5e6a1b4066b386b353325051be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3837ef2345c3439ebb5bcc00c82b7c39"
I20260812 11:18:43.869272 25646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:43.870484 25646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d50ce5e6a1b4066b386b353325051be in term 1.
I20260812 11:18:43.870846 25454 leader_election.cc:302] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3837ef2345c3439ebb5bcc00c82b7c39, 7d50ce5e6a1b4066b386b353325051be; no voters:
I20260812 11:18:43.871331 25352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d50ce5e6a1b4066b386b353325051be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7335a81b5ee54251b1fda900e91ceb88"
I20260812 11:18:43.871450 25352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:43.872596 25352 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d50ce5e6a1b4066b386b353325051be in term 1.
I20260812 11:18:43.875070 25747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:43.875201 25747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [term 1 LEADER]: Becoming Leader. State: Replica: 7d50ce5e6a1b4066b386b353325051be, State: Running, Role: LEADER
I20260812 11:18:43.875569 25747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } }
I20260812 11:18:43.877454 25747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d50ce5e6a1b4066b386b353325051be. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.877615 25747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:43.878072 25804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:43.879050 25804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:43.885735 25646 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [term 1 FOLLOWER]: Refusing update from remote peer 7d50ce5e6a1b4066b386b353325051be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:43.886499 25795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [LEADER]: Connected to new peer: Peer: permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:43.890298 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d50ce5e6a1b4066b386b353325051be. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.890409 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.893636 25352 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [term 1 FOLLOWER]: Refusing update from remote peer 7d50ce5e6a1b4066b386b353325051be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:43.894227 25795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:43.897811 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d50ce5e6a1b4066b386b353325051be. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.897928 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.908378 25795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.908517 25795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:43.912181 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.912297 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.913034 25806 mvcc.cc:205] Tried to move back new op lower bound from 7317641313828458496 to 7317641313797758976. Current Snapshot: MvccSnapshot[applied={T|T < 7317641313828458496}]
I20260812 11:18:43.914044 25747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.914155 25747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:43.915151 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d50ce5e6a1b4066b386b353325051be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7335a81b5ee54251b1fda900e91ceb88" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43303 } } peers { permanent_uuid: "7d50ce5e6a1b4066b386b353325051be" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34013 } } peers { permanent_uuid: "3837ef2345c3439ebb5bcc00c82b7c39" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 37485 } } }
I20260812 11:18:43.915284 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:18:43.916086 25807 mvcc.cc:205] Tried to move back new op lower bound from 7317641313828458496 to 7317641313797758976. Current Snapshot: MvccSnapshot[applied={T|T < 7317641313828458496}]
I20260812 11:18:43.916997 25804 catalog_manager.cc:1347] Generated new cluster ID: e84d2bd3dc8646d68562a1592bbd7a5b
I20260812 11:18:43.917065 25804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:43.926312 25809 mvcc.cc:205] Tried to move back new op lower bound from 7317641313828458496 to 7317641313797758976. Current Snapshot: MvccSnapshot[applied={T|T < 7317641313828458496}]
I20260812 11:18:43.959443 25804 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:18:43.960642 25804 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:43.973608 25804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d50ce5e6a1b4066b386b353325051be: Generated new TSK 0
I20260812 11:18:43.974390 25804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:18:44.770138 25744 catalog_manager.cc:1259] Loaded cluster ID: e84d2bd3dc8646d68562a1592bbd7a5b
I20260812 11:18:44.770218 25744 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: loading cluster ID for follower catalog manager: success
I20260812 11:18:44.771982 25744 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: acquiring CA information for follower catalog manager: success
I20260812 11:18:44.773114 25744 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3837ef2345c3439ebb5bcc00c82b7c39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:18:44.798548 25791 catalog_manager.cc:1259] Loaded cluster ID: e84d2bd3dc8646d68562a1592bbd7a5b
I20260812 11:18:44.798614 25791 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: loading cluster ID for follower catalog manager: success
I20260812 11:18:44.800395 25791 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: acquiring CA information for follower catalog manager: success
I20260812 11:18:44.801489 25791 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7335a81b5ee54251b1fda900e91ceb88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:18:45.038950 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 25269
I20260812 11:18:45.051252 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 25419
I20260812 11:18:45.065439 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 25550
2026-08-12T11:18:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3254 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:18:45 dist-test-slave-gjrz krb5kdc[25833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:18:45 dist-test-slave-gjrz krb5kdc[25833](info): set up 2 sockets
Aug 12 11:18:45 dist-test-slave-gjrz krb5kdc[25833](info): commencing operation
krb5kdc: starting...
W20260812 11:18:46.273288 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:18:46.387013 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41771
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:46.492429 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:46.492661 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:46.492717 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:46.496558 25862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:46.496611 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:46.496637 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:46.496658 25862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:46.496686 25862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:46.501699 25862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:46.502648 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:46.503607 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:46.511487 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:46.511492 25869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:46.511492 25873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:46.511586 25862 server_base.cc:1034] running on GCE node
I20260812 11:18:46.512300 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:46.513265 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:46.514425 25862 hybrid_clock.cc:648] HybridClock initialized: now 1786533526514408 us; error 25 us; skew 500 ppm
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533526, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:46.521216 25862 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:46.522028 25862 webserver.cc:460] Webserver started at http://127.30.255.190:43439/ using document root <none> and password file <none>
I20260812 11:18:46.522570 25862 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:46.522639 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:46.522804 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:46.524706 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "c6af740bf5a448a9a0c675df96e6b0a4"
format_stamp: "Formatted at 2026-08-12 11:18:46 on dist-test-slave-gjrz"
server_key: "13de1871a04c46f830cbacedb0266b21"
server_key_iv: "ccbc0f8b70696e6530de2434d830c610"
server_key_version: "encryptionkey@0"
I20260812 11:18:46.525197 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "c6af740bf5a448a9a0c675df96e6b0a4"
format_stamp: "Formatted at 2026-08-12 11:18:46 on dist-test-slave-gjrz"
server_key: "13de1871a04c46f830cbacedb0266b21"
server_key_iv: "ccbc0f8b70696e6530de2434d830c610"
server_key_version: "encryptionkey@0"
I20260812 11:18:46.528584 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 11:18:46.530694 25900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:46.531843 25862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:18:46.531945 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "c6af740bf5a448a9a0c675df96e6b0a4"
format_stamp: "Formatted at 2026-08-12 11:18:46 on dist-test-slave-gjrz"
server_key: "13de1871a04c46f830cbacedb0266b21"
server_key_iv: "ccbc0f8b70696e6530de2434d830c610"
server_key_version: "encryptionkey@0"
I20260812 11:18:46.532037 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:46.556075 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:46.558872 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:46.559031 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:46.565575 25862 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41771
I20260812 11:18:46.565599 25998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41771 every 8 connection(s)
I20260812 11:18:46.566425 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:46.569027 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:46.571985 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 25862
I20260812 11:18:46.572103 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:18:46.572357 31742 external_mini_cluster.cc:1428] Setting key 39f4325b8a666cd21ae186c79a0c410b
I20260812 11:18:46.574615 25999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4: Bootstrap starting.
I20260812 11:18:46.577152 25999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:46.577955 25999 log.cc:824] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:46.579982 25999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4: No bootstrap required, opened a new log
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[25833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:46.583329 25999 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } }
I20260812 11:18:46.583582 25999 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:46.583660 25999 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6af740bf5a448a9a0c675df96e6b0a4, State: Initialized, Role: FOLLOWER
I20260812 11:18:46.584131 25999 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } }
I20260812 11:18:46.584267 25999 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:46.584329 25999 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:46.584421 25999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:46.585392 25999 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } }
I20260812 11:18:46.585731 25999 leader_election.cc:302] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6af740bf5a448a9a0c675df96e6b0a4; no voters:
I20260812 11:18:46.586037 25999 leader_election.cc:288] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:18:46.586114 26006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:46.586350 26006 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [term 1 LEADER]: Becoming Leader. State: Replica: c6af740bf5a448a9a0c675df96e6b0a4, State: Running, Role: LEADER
I20260812 11:18:46.586690 26006 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } }
I20260812 11:18:46.586882 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:46.573568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.30.255.190:41771)
0812 11:18:46.573959 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:46.573977 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:46.574017 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:18:46.574798 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:46.574821 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:46.575207 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:46.575574 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:46.576221 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:46.577655 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:46.578576 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:46.579228 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:46.579417 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:46.581604 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:46.581610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:46.581626 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:46.581655 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:46.584025 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:46.584448 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:46.584454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:46.584460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:46.584603 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:46.584831 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:46.584834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:46.584836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:46.585309 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:46.585486 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:46.585715 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 11:18:46.585995 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":101,"threads_started":1}
I20260812 11:18:46.587195 25999 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:46.588301 26012 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6af740bf5a448a9a0c675df96e6b0a4. Latest consensus state: current_term: 1 leader_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } } }
I20260812 11:18:46.588443 26012 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:46.588301 26009 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6af740bf5a448a9a0c675df96e6b0a4" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41771 } } }
I20260812 11:18:46.588696 26009 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:46.588824 26027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:46.589905 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:18:46.591703 26027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:46.596773 26027 catalog_manager.cc:1347] Generated new cluster ID: b2598bdfe11345239bc540f52dedd38a
I20260812 11:18:46.596822 26027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:46.605765 26027 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:18:46.606559 26027 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:46.614851 26027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6af740bf5a448a9a0c675df96e6b0a4: Generated new TSK 0
I20260812 11:18:46.615417 26027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:18:46.645251 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 25862
2026-08-12T11:18:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1589 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[26057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[26057](info): set up 2 sockets
Aug 12 11:18:46 dist-test-slave-gjrz krb5kdc[26057](info): commencing operation
krb5kdc: starting...
W20260812 11:18:47.850016 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:18:47 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:18:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:18:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:18:47.962302 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:48.068903 26085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:48.069139 26085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:48.069193 26085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:48.073050 26085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:48.073096 26085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:48.073122 26085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:48.073143 26085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:48.073171 26085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:48.078087 26085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:48.078974 26085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:48.079917 26085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:48.087646 26091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.087674 26092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.087771 26094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.087853 26085 server_base.cc:1034] running on GCE node
I20260812 11:18:48.088428 26085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:48.089411 26085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:48.090560 26085 hybrid_clock.cc:648] HybridClock initialized: now 1786533528090542 us; error 29 us; skew 500 ppm
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:48.093050 26085 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:48.093822 26085 webserver.cc:460] Webserver started at http://127.30.255.190:44951/ using document root <none> and password file <none>
I20260812 11:18:48.094301 26085 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:48.094355 26085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:48.094506 26085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:48.096395 26085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "0e7f92da31c7480e97fa18607170b0ab"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "63a06d4a3a967f48afe9af17cff0c8a3"
server_key_iv: "51f5092d6f96fdd0e6ccff2ceb8acdd0"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.096933 26085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "0e7f92da31c7480e97fa18607170b0ab"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "63a06d4a3a967f48afe9af17cff0c8a3"
server_key_iv: "51f5092d6f96fdd0e6ccff2ceb8acdd0"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.100378 26085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:18:48.102546 26107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.103730 26085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:18:48.103827 26085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0e7f92da31c7480e97fa18607170b0ab"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "63a06d4a3a967f48afe9af17cff0c8a3"
server_key_iv: "51f5092d6f96fdd0e6ccff2ceb8acdd0"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.103906 26085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:48.126041 26085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:48.128592 26085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:48.128731 26085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:48.134924 26085 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41071
I20260812 11:18:48.135164 26229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41071 every 8 connection(s)
I20260812 11:18:48.135842 26085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:48.137447 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 26085
I20260812 11:18:48.137576 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:18:48.137842 31742 external_mini_cluster.cc:1428] Setting key 498a476010bc556285c3853de5dae289
I20260812 11:18:48.138505 26230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:48.144001 26230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap starting.
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:48.146122 26230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:48.146852 26230 log.cc:824] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:48.148670 26230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: No bootstrap required, opened a new log
I20260812 11:18:48.150606 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.139155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.30.255.190:41071)
0812 11:18:48.139594 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.139610 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.139644 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:18:48.140283 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.140299 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.140597 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.140893 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:48.141685 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.142689 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.143571 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.144035 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.144202 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.146406 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:48.146411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:48.146424 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:48.146449 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:48.148100 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.148487 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.148494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.148498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.148615 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.148870 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.148873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.148875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.149238 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:48.149410 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.149687 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.149946 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":136,"threads_started":1}
I20260812 11:18:48.152343 26230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:48.152588 26230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:48.152695 26230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Initialized, Role: FOLLOWER
I20260812 11:18:48.153090 26230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:48.153210 26230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:48.153270 26230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:48.153362 26230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:48.154155 26230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:48.154450 26230 leader_election.cc:302] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7f92da31c7480e97fa18607170b0ab; no voters:
I20260812 11:18:48.154713 26230 leader_election.cc:288] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:18:48.154790 26237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:48.154991 26237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 LEADER]: Becoming Leader. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Running, Role: LEADER
I20260812 11:18:48.155344 26237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:48.155838 26230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:48.156900 26241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e7f92da31c7480e97fa18607170b0ab. Latest consensus state: current_term: 1 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:48.157032 26241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:48.157496 26252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:48.156905 26238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:48.158329 26238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:48.160722 26252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:48.165796 26252 catalog_manager.cc:1347] Generated new cluster ID: afffc2a418fe45ddafc77f0c3a2492fa
I20260812 11:18:48.165856 26252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:48.197932 26252 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:18:48.199164 26252 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:48.203406 26252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Generated new TSK 0
I20260812 11:18:48.204135 26252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:18:48.239233 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:48.345906 26278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:48.346131 26278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:48.346172 26278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:48.350138 26278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:48.350188 26278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:48.350253 26278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:18:48.355167 26278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:48.356046 26278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:48.356968 26278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:48.365746 26288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.365725 26292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.365808 26278 server_base.cc:1034] running on GCE node
W20260812 11:18:48.365738 26286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.366287 26278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:48.366842 26278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:48.367987 26278 hybrid_clock.cc:648] HybridClock initialized: now 1786533528367959 us; error 38 us; skew 500 ppm
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:48.370456 26278 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:18:48.371230 26278 webserver.cc:460] Webserver started at http://127.30.255.129:45157/ using document root <none> and password file <none>
I20260812 11:18:48.371743 26278 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:48.371807 26278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:48.371984 26278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:48.373821 26278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "d71bbfd470b64cdda094f38a14289662"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "55b4a459a0dcb627292c03538019618d"
server_key_iv: "8df9844ae5e0be7eafb18772bbd5b5cf"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.374367 26278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "d71bbfd470b64cdda094f38a14289662"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "55b4a459a0dcb627292c03538019618d"
server_key_iv: "8df9844ae5e0be7eafb18772bbd5b5cf"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.377707 26278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 11:18:48.379916 26304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.381083 26278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 11:18:48.381174 26278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "d71bbfd470b64cdda094f38a14289662"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "55b4a459a0dcb627292c03538019618d"
server_key_iv: "8df9844ae5e0be7eafb18772bbd5b5cf"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.381253 26278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:48.390903 26278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:48.393661 26278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:48.393816 26278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:48.394340 26278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:48.395052 26278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:48.395097 26278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.395135 26278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:48.395162 26278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.403035 26278 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41945
I20260812 11:18:48.403057 26516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41945 every 8 connection(s)
I20260812 11:18:48.403837 26278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:18:48.404495 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 26278
I20260812 11:18:48.404641 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:18:48.404934 31742 external_mini_cluster.cc:1428] Setting key 7f9e8e738af69c0d03062979aa334ba7
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:18:48.416692 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.405550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:54143 (local address 127.30.255.190:41071)
0812 11:18:48.405676 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.405680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.406529 (+ 849us) server_negotiation.cc:407] Connection header received
0812 11:18:48.407879 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.407882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.407935 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.408006 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:48.409374 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.409896 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.410993 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.411313 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.411405 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.413558 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:48.413561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:48.413564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:48.413590 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:48.414838 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.415313 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.415316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.415318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.415400 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.415706 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.415709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.415711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.415880 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:48.415963 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.416412 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.416536 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:18:48.417376 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.405900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:54143)
0812 11:18:48.406388 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.406439 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.407675 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.408182 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.408190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.408645 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:48.409234 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.409253 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.410004 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.410007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.410877 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.410887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.411451 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.411455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.411616 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.412094 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:48.412109 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:48.413409 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:48.414941 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.414947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.414960 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.415208 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.415481 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.415484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.415486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.415619 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.415976 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:48.415982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:48.416199 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.416353 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.416648 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:18:48.418538 26518 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:48.418747 26518 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:48.419174 26518 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:48.420672 26147 ts_manager.cc:194] Registered new tserver with Master: d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945)
I20260812 11:18:48.422210 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:54143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:18:48.431376 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:48.537292 26524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:48.537513 26524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:48.537552 26524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:48.541385 26524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:48.541426 26524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:48.541478 26524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:18:48.546378 26524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:48.547223 26524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:48.548239 26524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:48.557188 26538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.557278 26531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.557416 26524 server_base.cc:1034] running on GCE node
W20260812 11:18:48.557199 26534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.557821 26524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:48.558435 26524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:48.559587 26524 hybrid_clock.cc:648] HybridClock initialized: now 1786533528559564 us; error 32 us; skew 500 ppm
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:48.562150 26524 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:18:48.562978 26524 webserver.cc:460] Webserver started at http://127.30.255.130:35789/ using document root <none> and password file <none>
I20260812 11:18:48.563544 26524 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:48.563609 26524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:48.563804 26524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:48.565711 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "25a84145090641418c9fca266ba9e8d8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "37da1c26f2cf110590e7fd569a1d1c17"
server_key_iv: "cce5c57db5d59d4c82dbf5e80486553f"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.566231 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "25a84145090641418c9fca266ba9e8d8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "37da1c26f2cf110590e7fd569a1d1c17"
server_key_iv: "cce5c57db5d59d4c82dbf5e80486553f"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.569612 26524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:18:48.571774 26551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.572855 26524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:18:48.572957 26524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "25a84145090641418c9fca266ba9e8d8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "37da1c26f2cf110590e7fd569a1d1c17"
server_key_iv: "cce5c57db5d59d4c82dbf5e80486553f"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.573057 26524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:48.599439 26524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:48.602206 26524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:48.602373 26524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:48.602909 26524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:48.603691 26524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:48.603739 26524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.603787 26524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:48.603812 26524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.611790 26524 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37791
I20260812 11:18:48.611814 26759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37791 every 8 connection(s)
I20260812 11:18:48.612563 26524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:18:48.616595 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 26524
I20260812 11:18:48.616703 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:18:48.616945 31742 external_mini_cluster.cc:1428] Setting key 1df0360cd8e53b2fbacdd77cb037363d
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:48.625243 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.614141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:53701 (local address 127.30.255.190:41071)
0812 11:18:48.614272 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.614276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.615016 (+ 740us) server_negotiation.cc:407] Connection header received
0812 11:18:48.616355 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.616358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.616413 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.616484 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:48.617718 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.618301 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.619428 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.619650 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.619735 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.622075 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:48.622078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:48.622080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:48.622112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:48.623375 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.623858 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.623861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.623863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.623944 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.624253 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.624256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.624258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.624430 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:48.624513 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.624964 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.625086 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:18:48.625932 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.614466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:53701)
0812 11:18:48.614885 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.614924 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.616149 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.616665 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.616674 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.617048 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:48.617577 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.617595 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.618425 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.618428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.619318 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.619328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.619790 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.619794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.619942 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.620418 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:48.620433 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:48.621926 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:48.623482 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.623487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.623501 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.623758 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.624037 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.624039 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.624041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.624163 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.624526 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:48.624532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:48.624744 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.624902 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.625199 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:18:48.627116 26763 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:48.627368 26763 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:48.627873 26763 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:48.628854 26147 ts_manager.cc:194] Registered new tserver with Master: 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:18:48.629431 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:53701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:18:48.642115 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:48.747645 26774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:48.747864 26774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:48.747905 26774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:48.751731 26774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:48.751776 26774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:48.751839 26774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:18:48.756784 26774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:48.757692 26774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:48.758587 26774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:48.767444 26788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.767457 26792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:48.767478 26787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:48.767695 26774 server_base.cc:1034] running on GCE node
I20260812 11:18:48.768064 26774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:48.768605 26774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:48.769758 26774 hybrid_clock.cc:648] HybridClock initialized: now 1786533528769743 us; error 28 us; skew 500 ppm
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:48.772270 26774 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:18:48.773011 26774 webserver.cc:460] Webserver started at http://127.30.255.131:38277/ using document root <none> and password file <none>
I20260812 11:18:48.773504 26774 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:48.773566 26774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:48.773737 26774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:18:48.775638 26774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "a40fd19f295c2fe7b6bd014192698e09"
server_key_iv: "b26e888c7b9438b390b8b20ff0d3fc47"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.776155 26774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "a40fd19f295c2fe7b6bd014192698e09"
server_key_iv: "b26e888c7b9438b390b8b20ff0d3fc47"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.779633 26774 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:18:48.781757 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.782835 26774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:18:48.782936 26774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "a40fd19f295c2fe7b6bd014192698e09"
server_key_iv: "b26e888c7b9438b390b8b20ff0d3fc47"
server_key_version: "encryptionkey@0"
I20260812 11:18:48.783020 26774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:48.796569 26774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:48.799247 26774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:48.799432 26774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:48.799927 26774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:48.800647 26774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:48.800693 26774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.800740 26774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:48.800768 26774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:48.809046 26774 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:43407
I20260812 11:18:48.809074 27049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:43407 every 8 connection(s)
I20260812 11:18:48.809794 26774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:18:48.817065 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 26774
I20260812 11:18:48.817159 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:18:48.817425 31742 external_mini_cluster.cc:1428] Setting key 8e25fbb5037605cd9c972b6bb843a423
Aug 12 11:18:48 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533528, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:48.821900 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.811386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:44513 (local address 127.30.255.190:41071)
0812 11:18:48.811513 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.811517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.812313 (+ 796us) server_negotiation.cc:407] Connection header received
0812 11:18:48.813564 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.813567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.813630 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.813704 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:48.814769 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.815328 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.816416 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.816644 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.816731 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.819140 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:48.819144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:48.819146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:48.819172 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:48.820312 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.820783 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.820785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.820786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.820850 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.821092 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.821094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.821095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.821239 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:48.821306 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.821692 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.821777 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:18:48.822603 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.811727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:44513)
0812 11:18:48.812177 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.812215 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.813364 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.813874 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.813882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.814202 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:48.814626 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.814638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.815443 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.815446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.816293 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.816303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.816778 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.816782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.816938 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.817443 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:48.817459 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:48.818982 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:48.820435 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.820440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.820450 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.820704 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.820907 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:48.820909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:48.820910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:48.821024 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:48.821308 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:48.821312 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:48.821512 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.821662 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.821894 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
I20260812 11:18:48.823582 27053 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:48.823772 27053 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:48.824178 27053 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:48.825089 26147 ts_manager.cc:194] Registered new tserver with Master: f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407)
I20260812 11:18:48.825616 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:44513
I20260812 11:18:48.830826 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:18:48.838909 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.832606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.30.255.190:41071)
0812 11:18:48.832716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.832720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.832894 (+ 174us) server_negotiation.cc:407] Connection header received
0812 11:18:48.833013 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.833015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.833063 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.833123 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:48.833759 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.834233 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.835015 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.835224 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.835357 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.836458 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:48.836462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:48.836465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:48.836488 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:48.837450 (+ 962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.837820 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.837824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.837825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.837895 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:48.838132 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:48.838135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:48.838137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:48.838331 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:48.838444 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.838675 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.838774 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:18:48.841200 26147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:18:48.844002 26147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:18:48.856993 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.851683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43407 (local address 127.0.0.1:50212)
0812 11:18:48.852435 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.852447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.852518 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.853186 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.853189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.853205 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:48.853311 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.853317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854240 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.854243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.855487 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.855498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.856502 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.856505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.856616 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.856653 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.856749 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.856861 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":17,"thread_start_us":79,"threads_started":1}
I20260812 11:18:48.857002 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.852043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.30.255.131:43407)
0812 11:18:48.852733 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.852745 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.852764 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:18:48.852824 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.852829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.852992 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.853121 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:48.853423 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854116 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.855623 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.856400 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.856506 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.856629 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.856787 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.856875 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":122,"threads_started":1}
I20260812 11:18:48.857335 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.851418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.0.0.1:60272)
0812 11:18:48.852113 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.852162 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.852280 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.853014 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.853027 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.853060 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:48.853277 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.853286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854661 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.854665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.855840 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.855850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.856855 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.856858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.856950 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.856972 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.857020 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.857203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":98,"spinlock_wait_cycles":2816,"thread_start_us":95,"threads_started":1}
I20260812 11:18:48.857461 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.851289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37791 (local address 127.0.0.1:34174)
0812 11:18:48.852112 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.852160 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.852322 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.853013 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.853019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.853057 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:48.853231 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.853240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854682 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.854685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.855731 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.855740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.857020 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.857024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.857117 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.857139 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.857204 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.857293 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":66,"spinlock_wait_cycles":8960,"thread_start_us":60,"threads_started":1}
I20260812 11:18:48.857478 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.851450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.30.255.130:37791)
0812 11:18:48.851886 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.851903 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.852249 (+ 346us) server_negotiation.cc:407] Connection header received
0812 11:18:48.852478 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.852484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.852682 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.852810 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:48.853358 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854516 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.855843 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.856721 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.856842 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.856964 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.857247 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.857343 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":122,"threads_started":1}
I20260812 11:18:48.858098 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.852049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.30.255.129:41945)
0812 11:18:48.852562 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.852574 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.852595 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:18:48.852665 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.852671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.852858 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.853008 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:48.853381 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.854516 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.855959 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.856721 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.857797 (+ 1076us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.857892 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.857929 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.857995 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":143,"threads_started":1}
I20260812 11:18:48.860383 26911 tablet_service.cc:1467] Processing CreateTablet for tablet d43605558de84f63aab3011b4e1c8a69 (DEFAULT_TABLE table=test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:48.860743 26633 tablet_service.cc:1467] Processing CreateTablet for tablet d43605558de84f63aab3011b4e1c8a69 (DEFAULT_TABLE table=test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:48.860764 26396 tablet_service.cc:1467] Processing CreateTablet for tablet d43605558de84f63aab3011b4e1c8a69 (DEFAULT_TABLE table=test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:48.861565 26911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43605558de84f63aab3011b4e1c8a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:48.861590 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43605558de84f63aab3011b4e1c8a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:48.861670 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43605558de84f63aab3011b4e1c8a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:48.867691 27078 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Bootstrap starting.
I20260812 11:18:48.868963 27081 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Bootstrap starting.
I20260812 11:18:48.870594 27078 tablet_bootstrap.cc:654] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:48.870779 27079 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Bootstrap starting.
I20260812 11:18:48.871421 27081 tablet_bootstrap.cc:654] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:48.871517 27078 log.cc:824] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:48.872179 27081 log.cc:824] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:48.872997 27079 tablet_bootstrap.cc:654] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:48.873618 27079 log.cc:824] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:48.873901 27078 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: No bootstrap required, opened a new log
I20260812 11:18:48.873901 27081 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: No bootstrap required, opened a new log
I20260812 11:18:48.874090 27081 ts_tablet_manager.cc:1397] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:18:48.874090 27078 ts_tablet_manager.cc:1397] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260812 11:18:48.875159 27079 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: No bootstrap required, opened a new log
I20260812 11:18:48.875349 27079 ts_tablet_manager.cc:1397] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:18:48.877843 27081 raft_consensus.cc:348] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.877843 27078 raft_consensus.cc:348] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.878098 27081 raft_consensus.cc:374] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:48.878098 27078 raft_consensus.cc:374] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:48.878154 27081 raft_consensus.cc:729] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71bbfd470b64cdda094f38a14289662, State: Initialized, Role: FOLLOWER
I20260812 11:18:48.878154 27078 raft_consensus.cc:729] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a84145090641418c9fca266ba9e8d8, State: Initialized, Role: FOLLOWER
I20260812 11:18:48.878108 27079 raft_consensus.cc:348] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.878341 27079 raft_consensus.cc:374] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:48.878407 27079 raft_consensus.cc:729] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1af60e8d1634391ab2fb73c4cd9aab8, State: Initialized, Role: FOLLOWER
I20260812 11:18:48.878611 27081 consensus_queue.cc:260] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.878611 27078 consensus_queue.cc:260] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.878788 27079 consensus_queue.cc:260] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.879585 27081 ts_tablet_manager.cc:1428] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:18:48.879583 27078 ts_tablet_manager.cc:1428] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260812 11:18:48.879727 27079 ts_tablet_manager.cc:1428] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 11:18:48.879910 26518 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
I20260812 11:18:48.880019 27053 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
I20260812 11:18:48.880496 26763 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
I20260812 11:18:48.889081 27088 raft_consensus.cc:482] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:48.889211 27088 raft_consensus.cc:504] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.890064 27088 leader_election.cc:288] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945), f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407)
I20260812 11:18:48.893503 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.890445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:46819 (local address 127.30.255.129:41945)
0812 11:18:48.890533 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.890537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.890548 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:18:48.890603 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.890606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.890648 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.890726 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:48.890965 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.891577 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.892597 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.892983 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.893067 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.893115 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.893327 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.893395 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:18:48.893527 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.890329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:52733 (local address 127.30.255.131:43407)
0812 11:18:48.890481 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.890485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.890724 (+ 239us) server_negotiation.cc:407] Connection header received
0812 11:18:48.890835 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.890838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.890878 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.890936 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:48.891191 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.891695 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.892631 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.893007 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.893126 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.893172 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.893372 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.893438 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 11:18:48.893513 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.890234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.30.255.130:46819)
0812 11:18:48.890364 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.890379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.890453 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.890732 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.890735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.890753 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:48.890858 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.890865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.891684 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.891688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.892466 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.892476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.893090 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.893093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.893194 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.893234 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.893290 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.893350 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260812 11:18:48.893543 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.890265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43407 (local address 127.30.255.130:52733)
0812 11:18:48.890638 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.890651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:48.890738 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:48.890963 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:48.890966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:48.890982 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:48.891102 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.891108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.891791 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.891794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.892515 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:48.892524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.893135 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.893139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:48.893254 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.893272 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 11:18:48.893344 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:18:48.893406 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":18,"thread_start_us":97,"threads_started":1}
I20260812 11:18:48.894145 26963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43605558de84f63aab3011b4e1c8a69" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" is_pre_election: true
I20260812 11:18:48.894276 26428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43605558de84f63aab3011b4e1c8a69" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71bbfd470b64cdda094f38a14289662" is_pre_election: true
I20260812 11:18:48.894366 26963 raft_consensus.cc:2454] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 0.
I20260812 11:18:48.894513 26428 raft_consensus.cc:2454] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 0.
I20260812 11:18:48.894767 26553 leader_election.cc:302] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8; no voters:
I20260812 11:18:48.894979 27088 raft_consensus.cc:2781] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:18:48.895058 27088 raft_consensus.cc:482] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:18:48.895108 27088 raft_consensus.cc:3037] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:48.896003 27088 raft_consensus.cc:504] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.896267 27088 leader_election.cc:288] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 election: Requested vote from peers d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945), f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407)
I20260812 11:18:48.896551 26428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43605558de84f63aab3011b4e1c8a69" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71bbfd470b64cdda094f38a14289662"
I20260812 11:18:48.896600 26963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43605558de84f63aab3011b4e1c8a69" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
I20260812 11:18:48.896661 26428 raft_consensus.cc:3037] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:48.896685 26963 raft_consensus.cc:3037] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:48.897753 26428 raft_consensus.cc:2454] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 1.
I20260812 11:18:48.897753 26963 raft_consensus.cc:2454] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 1.
I20260812 11:18:48.898005 26552 leader_election.cc:302] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a84145090641418c9fca266ba9e8d8, d71bbfd470b64cdda094f38a14289662; no voters:
I20260812 11:18:48.898195 27088 raft_consensus.cc:2781] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:48.898371 27088 raft_consensus.cc:686] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 1 LEADER]: Becoming Leader. State: Replica: 25a84145090641418c9fca266ba9e8d8, State: Running, Role: LEADER
I20260812 11:18:48.898627 27088 consensus_queue.cc:237] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:48.901181 26144 catalog_manager.cc:5495] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a84145090641418c9fca266ba9e8d8 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "25a84145090641418c9fca266ba9e8d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } health_report { overall_health: UNKNOWN } } }
W20260812 11:18:48.904803 26519 tablet.cc:2369] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:48.917055 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.914255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.30.255.130:37791)
0812 11:18:48.914352 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:48.914355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:48.914368 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:18:48.914473 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:48.914475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:48.914518 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:48.914593 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:18:48.914766 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.915215 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.915970 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:48.916172 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:48.916238 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:48.916443 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:18:48.916779 (+ 336us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:18:48.916860 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:48.916885 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 11:18:48.916925 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 11:18:48.921959 26428 raft_consensus.cc:1261] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Refusing update from remote peer 25a84145090641418c9fca266ba9e8d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:48.921960 26963 raft_consensus.cc:1261] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Refusing update from remote peer 25a84145090641418c9fca266ba9e8d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:48.922578 27096 consensus_queue.cc:1035] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:48.922730 27088 consensus_queue.cc:1035] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:48.928875 27104 mvcc.cc:205] Tried to move back new op lower bound from 7317641334458576896 to 7317641334370918400. Current Snapshot: MvccSnapshot[applied={T|T < 7317641334458576896}]
I20260812 11:18:48.931583 27105 mvcc.cc:205] Tried to move back new op lower bound from 7317641334458576896 to 7317641334370918400. Current Snapshot: MvccSnapshot[applied={T|T < 7317641334458576896}]
I20260812 11:18:48.936929 26143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:48.937060 26143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:48.939126 26143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Sending DeleteTablet for 3 replicas of tablet d43605558de84f63aab3011b4e1c8a69
I20260812 11:18:48.939791 26633 tablet_service.cc:1514] Processing DeleteTablet for tablet d43605558de84f63aab3011b4e1c8a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:48 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:34174
I20260812 11:18:48.939839 26396 tablet_service.cc:1514] Processing DeleteTablet for tablet d43605558de84f63aab3011b4e1c8a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:48 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260812 11:18:48.939870 26911 tablet_service.cc:1514] Processing DeleteTablet for tablet d43605558de84f63aab3011b4e1c8a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:48 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:50212
I20260812 11:18:48.940336 27123 tablet_replica.cc:330] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: stopping tablet replica
I20260812 11:18:48.940361 27126 tablet_replica.cc:330] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: stopping tablet replica
I20260812 11:18:48.940376 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 26278
I20260812 11:18:48.951998 27123 raft_consensus.cc:2229] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:18:48.952459 27123 raft_consensus.cc:2258] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:48.953557 27127 tablet_replica.cc:330] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: stopping tablet replica
I20260812 11:18:48.954531 27123 ts_tablet_manager.cc:1905] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:48.957199 27123 ts_tablet_manager.cc:1918] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:48.957312 27123 log.cc:1196] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/d43605558de84f63aab3011b4e1c8a69
I20260812 11:18:48.957629 27123 ts_tablet_manager.cc:1939] T d43605558de84f63aab3011b4e1c8a69 P 25a84145090641418c9fca266ba9e8d8: Deleting consensus metadata
I20260812 11:18:48.958807 26117 catalog_manager.cc:4841] TS 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): tablet d43605558de84f63aab3011b4e1c8a69 (table test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]) successfully deleted
I20260812 11:18:48.958877 27127 raft_consensus.cc:2229] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:48.959187 27127 raft_consensus.cc:2258] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:48.960462 27127 ts_tablet_manager.cc:1905] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:48.962931 27127 ts_tablet_manager.cc:1918] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 11:18:48.962951 26108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:41945 (error 108)
I20260812 11:18:48.963032 27127 log.cc:1196] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/d43605558de84f63aab3011b4e1c8a69
I20260812 11:18:48.963238 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 26524
I20260812 11:18:48.970636 27127 ts_tablet_manager.cc:1939] T d43605558de84f63aab3011b4e1c8a69 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting consensus metadata
I20260812 11:18:48.972393 26112 catalog_manager.cc:4841] TS f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): tablet d43605558de84f63aab3011b4e1c8a69 (table test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]) successfully deleted
I20260812 11:18:48.978310 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:48.978041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.0.0.1:60280)
0812 11:18:48.978171 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:48.978236 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 11:18:48.978484 26108 catalog_manager.cc:4584] TS d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d43605558de84f63aab3011b4e1c8a69: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41945: connect: Connection refused (error 111)
I20260812 11:18:48.982848 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 26774
I20260812 11:18:48.996304 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 26085
I20260812 11:18:49.011191 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--webserver_interface=127.30.255.190
--webserver_port=44951
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:49.118134 27129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:49.118356 27129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:49.118392 27129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:49.122284 27129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:49.122331 27129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:49.122349 27129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:49.122363 27129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:49.122378 27129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:49.127363 27129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=44951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:49.128388 27129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:49.129364 27129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:49.142584 27146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.142614 27141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.142589 27143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:49.142756 27129 server_base.cc:1034] running on GCE node
I20260812 11:18:49.143487 27129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:49.144542 27129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:49.145692 27129 hybrid_clock.cc:648] HybridClock initialized: now 1786533529145671 us; error 30 us; skew 500 ppm
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:49.148304 27129 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:49.149194 27129 webserver.cc:460] Webserver started at http://127.30.255.190:44951/ using document root <none> and password file <none>
I20260812 11:18:49.149729 27129 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:49.149811 27129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:49.153535 27129 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 11:18:49.155546 27169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.156670 27129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:18:49.156785 27129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0e7f92da31c7480e97fa18607170b0ab"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "63a06d4a3a967f48afe9af17cff0c8a3"
server_key_iv: "51f5092d6f96fdd0e6ccff2ceb8acdd0"
server_key_version: "encryptionkey@0"
I20260812 11:18:49.157363 27129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:49.184818 27129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:49.187539 27129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:49.187706 27129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:49.194410 27129 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41071
I20260812 11:18:49.194671 27267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41071 every 8 connection(s)
I20260812 11:18:49.195501 27129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:49.197063 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 27129
I20260812 11:18:49.197479 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:41945
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=45157
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:49.199846 27268 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:18:49.201422 27268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap starting.
I20260812 11:18:49.217844 27268 log.cc:824] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:49.226143 27268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:18:49.226450 27268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap complete.
I20260812 11:18:49.230226 27268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:49.230607 27268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Initialized, Role: FOLLOWER
I20260812 11:18:49.231117 27268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:49.231240 27268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:49.231341 27268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:49.231439 27268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:18:49.233305 27268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:49.233611 27268 leader_election.cc:302] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7f92da31c7480e97fa18607170b0ab; no voters:
I20260812 11:18:49.233904 27268 leader_election.cc:288] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:18:49.234006 27275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:18:49.234189 27275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 LEADER]: Becoming Leader. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Running, Role: LEADER
I20260812 11:18:49.234467 27275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:49.235081 27268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:49.235370 27275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e7f92da31c7480e97fa18607170b0ab. Latest consensus state: current_term: 2 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:49.235484 27275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:49.235870 27276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:49.236035 27276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:49.239989 27304 catalog_manager.cc:1259] Loaded cluster ID: afffc2a418fe45ddafc77f0c3a2492fa
I20260812 11:18:49.240057 27304 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: loading cluster ID for follower catalog manager: success
W20260812 11:18:49.240958 27304 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260812 11:18:49.241040 27304 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260812 11:18:49.241096 27280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:49.243448 27280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:49.243927 27280 catalog_manager.cc:1259] Loaded cluster ID: afffc2a418fe45ddafc77f0c3a2492fa
I20260812 11:18:49.243985 27280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:18:49.245039 27280 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 11:18:49.249768 27280 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 11:18:49.254741 27280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:18:49.255484 27280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Loaded TSK: 0
I20260812 11:18:49.256623 27280 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:18:49.308740 27273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:49.308966 27273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:49.309024 27273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:49.313786 27273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:49.313844 27273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:49.313920 27273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:18:49.318882 27273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:41945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=45157
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:49.319811 27273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:49.320737 27273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:49.330701 27314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.330713 27315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.330780 27322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:49.331243 27273 server_base.cc:1034] running on GCE node
I20260812 11:18:49.331638 27273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:49.332259 27273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:49.333426 27273 hybrid_clock.cc:648] HybridClock initialized: now 1786533529333405 us; error 31 us; skew 500 ppm
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:49.336042 27273 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:18:49.336869 27273 webserver.cc:460] Webserver started at http://127.30.255.129:45157/ using document root <none> and password file <none>
I20260812 11:18:49.337401 27273 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:49.337483 27273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:49.341312 27273 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 11:18:49.343448 27339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.344714 27273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:18:49.344823 27273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "d71bbfd470b64cdda094f38a14289662"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "55b4a459a0dcb627292c03538019618d"
server_key_iv: "8df9844ae5e0be7eafb18772bbd5b5cf"
server_key_version: "encryptionkey@0"
I20260812 11:18:49.345323 27273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:49.364672 27273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:49.367581 27273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:49.367744 27273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:49.368302 27273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:49.369338 27354 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:18:49.373190 27273 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:18:49.373242 27273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260812 11:18:49.373286 27273 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:18:49.374825 27273 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:18:49.374881 27273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 11:18:49.375151 27354 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Bootstrap starting.
I20260812 11:18:49.385094 27273 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41945
I20260812 11:18:49.385109 27575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41945 every 8 connection(s)
I20260812 11:18:49.386219 27273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:18:49.394430 27354 log.cc:824] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:49.394550 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 27273
I20260812 11:18:49.395020 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:37791
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=35789
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:49.401844 27354 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:18:49.402262 27354 tablet_bootstrap.cc:492] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Bootstrap complete.
I20260812 11:18:49.402496 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.388250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:50399)
0812 11:18:49.388727 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.388767 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.389518 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.390571 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.390579 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.390971 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:49.391462 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.391477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.392713 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.392716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.393566 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.393575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.394257 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.394260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.394454 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.395028 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:49.395049 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:49.396819 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:49.399110 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.399116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.399129 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.399482 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.399893 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.399896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.399898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.400070 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.400759 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:49.400765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:49.401030 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.401210 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.401502 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260812 11:18:49.402534 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.388190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:50399 (local address 127.30.255.190:41071)
0812 11:18:49.388531 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.388551 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.388883 (+ 332us) server_negotiation.cc:407] Connection header received
0812 11:18:49.389911 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.389922 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.390202 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.390490 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:49.391625 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.392570 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.393681 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.394127 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.394270 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.397029 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:49.397036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:49.397053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:49.397086 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:49.398989 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.399603 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.399610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.399617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.399789 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.400183 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.400187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.400188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.400639 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:49.400780 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.401370 (+ 590us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.401669 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260812 11:18:49.403571 27354 ts_tablet_manager.cc:1397] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Time spent bootstrapping tablet: real 0.029s user 0.016s sys 0.012s
I20260812 11:18:49.404688 27581 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:49.404901 27581 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:49.405328 27581 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:49.406401 27354 raft_consensus.cc:348] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:49.406931 27354 raft_consensus.cc:729] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71bbfd470b64cdda094f38a14289662, State: Initialized, Role: FOLLOWER
I20260812 11:18:49.407449 27354 consensus_queue.cc:260] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } } peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } }
I20260812 11:18:49.407550 27199 ts_manager.cc:194] Registered new tserver with Master: d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945)
I20260812 11:18:49.408471 27354 ts_tablet_manager.cc:1428] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:18:49.410766 27199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:50399
I20260812 11:18:49.413210 27581 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:18:49.420147 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.410300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.30.255.129:41945)
0812 11:18:49.410637 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.410647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.411026 (+ 379us) server_negotiation.cc:407] Connection header received
0812 11:18:49.411200 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.411205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.411420 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.411547 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:49.412220 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.413039 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.414046 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.414562 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.414660 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.417051 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:49.417061 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:49.417074 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:49.417111 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:49.418499 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.418979 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.418982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.418984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.419050 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.419314 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.419317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.419319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.419522 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:49.419625 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.419923 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.420038 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":80,"threads_started":1}
I20260812 11:18:49.420274 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.410170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.0.0.1:60282)
0812 11:18:49.410887 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.410933 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.411067 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.411700 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.411705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.412014 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:49.412106 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.412112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.413150 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.413153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.413907 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.413918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.414777 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.414780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.414893 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.415412 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:49.415428 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:49.416925 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:49.418601 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.418610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.418616 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.418887 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.419124 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.419127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.419129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.419216 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.419618 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:49.419624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:49.419739 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.419872 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.420038 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":98,"threads_started":1}
I20260812 11:18:49.421226 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet d43605558de84f63aab3011b4e1c8a69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:48 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:60282
I20260812 11:18:49.421912 27597 tablet_replica.cc:330] stopping tablet replica
I20260812 11:18:49.422153 27597 raft_consensus.cc:2229] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:49.422349 27597 raft_consensus.cc:2258] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:49.423157 27597 ts_tablet_manager.cc:1905] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:49.426311 27597 ts_tablet_manager.cc:1918] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:49.426427 27597 log.cc:1196] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/d43605558de84f63aab3011b4e1c8a69
I20260812 11:18:49.426664 27597 ts_tablet_manager.cc:1939] T d43605558de84f63aab3011b4e1c8a69 P d71bbfd470b64cdda094f38a14289662: Deleting consensus metadata
I20260812 11:18:49.428135 27170 catalog_manager.cc:4841] TS d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945): tablet d43605558de84f63aab3011b4e1c8a69 (table test-table [id=5f31c9e292504ecb96b0bc3ca88cc46b]) successfully deleted
W20260812 11:18:49.511137 27588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:49.511407 27588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:49.511453 27588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:49.515309 27588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:49.515353 27588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:49.515405 27588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:18:49.520265 27588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:37791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=35789
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:49.521095 27588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:49.522033 27588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:49.530740 27610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:49.530836 27588 server_base.cc:1034] running on GCE node
W20260812 11:18:49.530751 27618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.530759 27612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:49.531445 27588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:49.532027 27588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:49.533181 27588 hybrid_clock.cc:648] HybridClock initialized: now 1786533529533164 us; error 29 us; skew 500 ppm
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:49.535737 27588 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:18:49.536562 27588 webserver.cc:460] Webserver started at http://127.30.255.130:35789/ using document root <none> and password file <none>
I20260812 11:18:49.537030 27588 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:49.537132 27588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:49.540647 27588 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:18:49.542479 27634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.543581 27588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:18:49.543681 27588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "25a84145090641418c9fca266ba9e8d8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "37da1c26f2cf110590e7fd569a1d1c17"
server_key_iv: "cce5c57db5d59d4c82dbf5e80486553f"
server_key_version: "encryptionkey@0"
I20260812 11:18:49.544157 27588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:49.575337 27588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:49.578089 27588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:49.578243 27588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:49.578785 27588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:49.579566 27588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:49.579614 27588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.579663 27588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:49.579691 27588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.587695 27588 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37791
I20260812 11:18:49.587750 27857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37791 every 8 connection(s)
I20260812 11:18:49.588574 27588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:18:49.592391 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 27588
I20260812 11:18:49.592768 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:43407
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=38277
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:49.601256 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.590157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:48693 (local address 127.30.255.190:41071)
0812 11:18:49.590285 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.590289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.591035 (+ 746us) server_negotiation.cc:407] Connection header received
0812 11:18:49.592347 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.592351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.592410 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.592486 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:49.593643 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.594163 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.595340 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.595738 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.595838 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.598065 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:49.598069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:49.598071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:49.598101 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:49.599388 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.599840 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.599844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.599846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.599930 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.600234 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.600237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.600239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.600417 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:49.600501 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.600961 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.601092 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:18:49.601987 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.590480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:48693)
0812 11:18:49.590897 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.590935 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.592139 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.592665 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.592672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.593046 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:49.593496 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.593509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.594280 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.594283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.595198 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.595210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.595895 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.595900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.596067 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.596561 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:49.596576 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:49.597904 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:49.599498 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.599504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.599517 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.599747 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.600010 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.600013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.600015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.600147 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.600514 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:49.600520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:49.600746 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.600903 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.601211 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":106,"threads_started":1}
I20260812 11:18:49.603055 27861 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:49.603307 27861 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:49.603765 27861 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:49.604744 27199 ts_manager.cc:194] Registered new tserver with Master: 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
I20260812 11:18:49.605477 27199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:48693
W20260812 11:18:49.701694 27869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:49.701930 27869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:49.701969 27869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:49.705760 27869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:49.705806 27869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:49.705857 27869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:18:49.710754 27869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:43407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=38277
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:49.711654 27869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:49.712517 27869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:49.721237 27881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.721254 27877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:49.721254 27876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:49.721429 27869 server_base.cc:1034] running on GCE node
I20260812 11:18:49.721899 27869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:49.722452 27869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:49.723605 27869 hybrid_clock.cc:648] HybridClock initialized: now 1786533529723588 us; error 26 us; skew 500 ppm
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:49.726020 27869 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:18:49.726790 27869 webserver.cc:460] Webserver started at http://127.30.255.131:38277/ using document root <none> and password file <none>
I20260812 11:18:49.727330 27869 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:49.727416 27869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:49.730963 27869 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:18:49.732892 27895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.733989 27869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:18:49.734093 27869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "a40fd19f295c2fe7b6bd014192698e09"
server_key_iv: "b26e888c7b9438b390b8b20ff0d3fc47"
server_key_version: "encryptionkey@0"
I20260812 11:18:49.734606 27869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:49.755878 27869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:49.758373 27869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:49.758529 27869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:49.759034 27869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:49.759806 27869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:49.759852 27869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.759891 27869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:49.759919 27869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:49.768028 27869 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:43407
I20260812 11:18:49.768067 28131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:43407 every 8 connection(s)
I20260812 11:18:49.768883 27869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:18:49.769584 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 27869
I20260812 11:18:49.777907 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.770200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.30.255.190:41071)
0812 11:18:49.770334 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.770339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.770623 (+ 284us) server_negotiation.cc:407] Connection header received
0812 11:18:49.770748 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.770750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.770801 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.770886 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:49.771710 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.772212 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.773342 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.773544 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.773617 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.774941 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:49.774944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:49.774947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:49.774973 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:49.776374 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.776817 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.776819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.776820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.776891 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.777155 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.777157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.777159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.777321 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:49.777458 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.777672 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.777756 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 11:18:49 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533529, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:18:49.781801 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.770426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:39955 (local address 127.30.255.190:41071)
0812 11:18:49.770621 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.770625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.771621 (+ 996us) server_negotiation.cc:407] Connection header received
0812 11:18:49.773055 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.773056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.773095 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.773145 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:49.774438 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.774950 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.776076 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.776412 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.776497 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.778813 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:49.778816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:49.778819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:49.778841 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:49.780269 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.780703 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.780705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.780706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.780776 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:49.781027 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:49.781029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:49.781030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:49.781172 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:49.781238 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.781603 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.781687 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":60,"threads_started":1}
I20260812 11:18:49.782486 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.770942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:39955)
0812 11:18:49.771466 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.771519 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.772864 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.773306 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.773316 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.773688 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:49.774293 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.774314 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.775075 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.775078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.775957 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.775967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.776543 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.776547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.776736 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.777201 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:49.777216 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:49.778670 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:49.780365 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.780369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.780379 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.780625 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.780837 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:49.780839 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:49.780840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:49.780958 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:49.781238 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:49.781242 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:49.781436 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.781572 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.781788 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":103,"threads_started":1}
I20260812 11:18:49.783406 28132 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41071
I20260812 11:18:49.783595 28132 heartbeater.cc:464] Registering TS with master...
I20260812 11:18:49.783977 28132 heartbeater.cc:511] Master 127.30.255.190:41071 requested a full tablet report, sending...
I20260812 11:18:49.784819 27199 ts_manager.cc:194] Registered new tserver with Master: f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407)
I20260812 11:18:49.785324 27199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:39955
I20260812 11:18:49.791426 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:18:49.795300 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.791941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.30.255.190:41071)
0812 11:18:49.792049 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.792054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.792210 (+ 156us) server_negotiation.cc:407] Connection header received
0812 11:18:49.792361 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.792365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.792446 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.792534 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:18:49.792741 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.793217 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.794028 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.794255 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.794339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.794625 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:18:49.794934 (+ 309us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:18:49.795031 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.795115 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.795168 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:18:49.795823 27199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:18:49.798795 27199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:18:49.805866 27437 tablet_service.cc:1467] Processing CreateTablet for tablet fa116e8dce024f9ea6720650d7849d96 (DEFAULT_TABLE table=test-table [id=0c2f23a705844f5888335405ebf400ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:49.806458 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa116e8dce024f9ea6720650d7849d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:49.809142 27354 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Bootstrap starting.
I20260812 11:18:49.810009 27354 tablet_bootstrap.cc:654] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:49.810057 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.804662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43407 (local address 127.0.0.1:58650)
0812 11:18:49.805019 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.805032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.805101 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.805837 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.805840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.805856 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:49.805943 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.805950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.807351 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.807355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.808516 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.808528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.809606 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.809609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.809737 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.809759 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.809824 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.809897 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":46}
I20260812 11:18:49.810269 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.804794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.30.255.131:43407)
0812 11:18:49.805404 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.805414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.805437 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:18:49.805503 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.805509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.805693 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.805842 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:49.806069 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.807211 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.808653 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.809494 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.809601 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.809715 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.809860 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.809944 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":145,"threads_started":1}
I20260812 11:18:49.810745 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.805137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.30.255.130:37791)
0812 11:18:49.805598 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:49.805607 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:49.805623 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:18:49.805680 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:49.805683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:49.805823 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:49.805964 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:49.806238 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.807339 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.808593 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.809172 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.809290 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.809415 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:49.810368 (+ 953us) server_negotiation.cc:299] Negotiation successful
0812 11:18:49.810472 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":114,"threads_started":1}
I20260812 11:18:49.809926 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:49.804963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37791 (local address 127.0.0.1:58076)
0812 11:18:49.805345 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:49.805357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:49.805437 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:49.806003 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:49.806005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:49.806024 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:49.806121 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.806128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.807595 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.807598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.808478 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:49.808488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:49.809482 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:49.809485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:49.809598 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:49.809667 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 11:18:49.809734 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:18:49.809801 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":7,"thread_start_us":108,"threads_started":1}
I20260812 11:18:49.811676 27354 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: No bootstrap required, opened a new log
I20260812 11:18:49.811765 27354 ts_tablet_manager.cc:1397] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 11:18:49.812134 27354 raft_consensus.cc:348] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.812227 27354 raft_consensus.cc:374] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:49.812263 27354 raft_consensus.cc:729] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71bbfd470b64cdda094f38a14289662, State: Initialized, Role: FOLLOWER
I20260812 11:18:49.812377 27354 consensus_queue.cc:260] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.812631 27354 ts_tablet_manager.cc:1428] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:18:49.813109 28002 tablet_service.cc:1467] Processing CreateTablet for tablet fa116e8dce024f9ea6720650d7849d96 (DEFAULT_TABLE table=test-table [id=0c2f23a705844f5888335405ebf400ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:49.813694 27719 tablet_service.cc:1467] Processing CreateTablet for tablet fa116e8dce024f9ea6720650d7849d96 (DEFAULT_TABLE table=test-table [id=0c2f23a705844f5888335405ebf400ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:18:49.814157 28002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa116e8dce024f9ea6720650d7849d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:49.814692 27719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa116e8dce024f9ea6720650d7849d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:18:49.821975 28154 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Bootstrap starting.
I20260812 11:18:49.822418 28153 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Bootstrap starting.
I20260812 11:18:49.824872 28153 tablet_bootstrap.cc:654] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:49.824872 28154 tablet_bootstrap.cc:654] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Neither blocks nor log segments found. Creating new log.
I20260812 11:18:49.825740 28154 log.cc:824] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:49.825740 28153 log.cc:824] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:49.827414 28154 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: No bootstrap required, opened a new log
I20260812 11:18:49.827427 28153 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: No bootstrap required, opened a new log
I20260812 11:18:49.827585 28154 ts_tablet_manager.cc:1397] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260812 11:18:49.827584 28153 ts_tablet_manager.cc:1397] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:18:49.830443 28153 raft_consensus.cc:348] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.830444 28154 raft_consensus.cc:348] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.830639 28153 raft_consensus.cc:374] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:49.830643 28154 raft_consensus.cc:374] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:18:49.830672 28153 raft_consensus.cc:729] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1af60e8d1634391ab2fb73c4cd9aab8, State: Initialized, Role: FOLLOWER
I20260812 11:18:49.830703 28154 raft_consensus.cc:729] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a84145090641418c9fca266ba9e8d8, State: Initialized, Role: FOLLOWER
I20260812 11:18:49.831076 28154 consensus_queue.cc:260] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.831072 28153 consensus_queue.cc:260] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:49.831725 28132 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
I20260812 11:18:49.831730 27861 heartbeater.cc:503] Master 127.30.255.190:41071 was elected leader, sending a full tablet report...
I20260812 11:18:49.832069 28154 ts_tablet_manager.cc:1428] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 11:18:49.832065 28153 ts_tablet_manager.cc:1428] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260812 11:18:49.839399 27862 tablet.cc:2369] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:18:49.887332 27582 tablet.cc:2369] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:50.006771 28159 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:50.006964 28159 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:50.008116 28159 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945)
I20260812 11:18:50.011492 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.008340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43407 (local address 127.30.255.130:43603)
0812 11:18:50.008460 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.008475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:50.008573 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:50.008819 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:50.008822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:50.008842 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:50.008958 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:50.008963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.009679 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.009682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:50.010447 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:50.010456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.011072 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.011076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:50.011185 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:50.011206 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:18:50.011289 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 11:18:50.011365 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 11:18:50.011492 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.008441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:43603 (local address 127.30.255.131:43407)
0812 11:18:50.008583 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:50.008588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:50.008606 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:18:50.008663 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:50.008666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:50.008722 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:50.008794 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:50.009069 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.009578 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.010573 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.010962 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.011066 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:50.011120 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:50.011308 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 11:18:50.011371 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:18:50.011920 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.008340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.30.255.130:44877)
0812 11:18:50.008786 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.008799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:50.008895 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:50.009148 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:50.009151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:50.009166 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:18:50.009278 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:50.009284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.009963 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.009966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:50.010718 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:50.010726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.011397 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.011401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:50.011493 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:50.011514 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:18:50.011723 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 11:18:50.011796 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":120,"thread_start_us":101,"threads_started":1}
I20260812 11:18:50.011956 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.008435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:44877 (local address 127.30.255.129:41945)
0812 11:18:50.008786 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:50.008791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:50.008868 (+ 77us) server_negotiation.cc:407] Connection header received
0812 11:18:50.008977 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:50.008980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:50.009037 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:50.009131 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:18:50.009380 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.009866 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.010831 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.011291 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.011402 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:50.011551 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:50.011758 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 11:18:50.011839 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":96,"threads_started":1}
I20260812 11:18:50.012166 28036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa116e8dce024f9ea6720650d7849d96" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" is_pre_election: true
I20260812 11:18:50.012406 28036 raft_consensus.cc:2454] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 0.
I20260812 11:18:50.012508 27481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa116e8dce024f9ea6720650d7849d96" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71bbfd470b64cdda094f38a14289662" is_pre_election: true
I20260812 11:18:50.012773 27481 raft_consensus.cc:2454] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 0.
I20260812 11:18:50.012852 27639 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8; no voters:
I20260812 11:18:50.013118 28159 raft_consensus.cc:2781] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:18:50.013198 28159 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:18:50.013229 28159 raft_consensus.cc:3037] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:50.014273 28159 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:50.014586 28159 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 election: Requested vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945)
I20260812 11:18:50.014896 27481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa116e8dce024f9ea6720650d7849d96" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71bbfd470b64cdda094f38a14289662"
I20260812 11:18:50.015017 27481 raft_consensus.cc:3037] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:50.015404 28036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa116e8dce024f9ea6720650d7849d96" candidate_uuid: "25a84145090641418c9fca266ba9e8d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
I20260812 11:18:50.015508 28036 raft_consensus.cc:3037] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:18:50.016058 27481 raft_consensus.cc:2454] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 1.
I20260812 11:18:50.016319 27637 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a84145090641418c9fca266ba9e8d8, d71bbfd470b64cdda094f38a14289662; no voters:
I20260812 11:18:50.016487 28159 raft_consensus.cc:2781] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:18:50.016614 28036 raft_consensus.cc:2454] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a84145090641418c9fca266ba9e8d8 in term 1.
I20260812 11:18:50.016659 28159 raft_consensus.cc:686] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 1 LEADER]: Becoming Leader. State: Replica: 25a84145090641418c9fca266ba9e8d8, State: Running, Role: LEADER
I20260812 11:18:50.016989 28159 consensus_queue.cc:237] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
W20260812 11:18:50.019671 28133 tablet.cc:2369] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:18:50.019906 27198 catalog_manager.cc:5495] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a84145090641418c9fca266ba9e8d8 (127.30.255.130). New cstate: current_term: 1 leader_uuid: "25a84145090641418c9fca266ba9e8d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } health_report { overall_health: UNKNOWN } } }
I20260812 11:18:50.042820 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.039906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.30.255.130:37791)
0812 11:18:50.040008 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:50.040011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:50.040023 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:18:50.040170 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:50.040172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:50.040220 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:50.040298 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:18:50.040454 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.040915 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.041648 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:50.041841 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:50.041917 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:50.042172 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:18:50.042535 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:18:50.042621 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:50.042648 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 11:18:50.042692 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:18:50.048085 27481 raft_consensus.cc:1261] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Refusing update from remote peer 25a84145090641418c9fca266ba9e8d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:50.048100 28036 raft_consensus.cc:1261] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Refusing update from remote peer 25a84145090641418c9fca266ba9e8d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:18:50.048732 28165 consensus_queue.cc:1035] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:50.048906 28159 consensus_queue.cc:1035] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:18:50.055366 28175 mvcc.cc:205] Tried to move back new op lower bound from 7317641339070222336 to 7317641338951999488. Current Snapshot: MvccSnapshot[applied={T|T < 7317641339070222336}]
I20260812 11:18:50.055696 28178 mvcc.cc:205] Tried to move back new op lower bound from 7317641339070222336 to 7317641338951999488. Current Snapshot: MvccSnapshot[applied={T|T < 7317641339070222336}]
I20260812 11:18:50.062315 27199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:50.062440 27199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:18:50.064591 27199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Sending DeleteTablet for 3 replicas of tablet fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:50.065235 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet fa116e8dce024f9ea6720650d7849d96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:50 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:60282
I20260812 11:18:50.065243 28002 tablet_service.cc:1514] Processing DeleteTablet for tablet fa116e8dce024f9ea6720650d7849d96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:50 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58650
I20260812 11:18:50.065266 27719 tablet_service.cc:1514] Processing DeleteTablet for tablet fa116e8dce024f9ea6720650d7849d96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:18:50 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:58076
I20260812 11:18:50.065668 28200 tablet_replica.cc:330] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: stopping tablet replica
I20260812 11:18:50.065775 28199 tablet_replica.cc:330] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: stopping tablet replica
I20260812 11:18:50.065834 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 27273
I20260812 11:18:50.065871 28199 raft_consensus.cc:2229] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:50.065946 28200 raft_consensus.cc:2229] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:18:50.065982 28199 raft_consensus.cc:2258] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:50.066190 28200 raft_consensus.cc:2258] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:50.074200 28200 ts_tablet_manager.cc:1905] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:50.076484 28200 ts_tablet_manager.cc:1918] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:50.076568 28200 log.cc:1196] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:50.076830 28200 ts_tablet_manager.cc:1939] T fa116e8dce024f9ea6720650d7849d96 P f1af60e8d1634391ab2fb73c4cd9aab8: Deleting consensus metadata
I20260812 11:18:50.077848 27171 catalog_manager.cc:4841] TS f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): tablet fa116e8dce024f9ea6720650d7849d96 (table test-table [id=0c2f23a705844f5888335405ebf400ed]) successfully deleted
I20260812 11:18:50.078243 28202 tablet_replica.cc:330] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: stopping tablet replica
I20260812 11:18:50.078500 28202 raft_consensus.cc:2229] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:18:50.078817 28202 raft_consensus.cc:2258] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:18:50.103140 28202 ts_tablet_manager.cc:1905] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:18:50.106020 28202 ts_tablet_manager.cc:1918] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:18:50.106132 28202 log.cc:1196] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:50.106415 28202 ts_tablet_manager.cc:1939] T fa116e8dce024f9ea6720650d7849d96 P 25a84145090641418c9fca266ba9e8d8: Deleting consensus metadata
I20260812 11:18:50.107755 27174 catalog_manager.cc:4841] TS 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): tablet fa116e8dce024f9ea6720650d7849d96 (table test-table [id=0c2f23a705844f5888335405ebf400ed]) successfully deleted
W20260812 11:18:50.109397 27170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:41945 (error 108)
I20260812 11:18:50.109894 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 27588
I20260812 11:18:50.112438 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.112154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:41945 (local address 127.0.0.1:49712)
0812 11:18:50.112287 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.112353 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 11:18:50.112586 27170 catalog_manager.cc:4584] TS d71bbfd470b64cdda094f38a14289662 (127.30.255.129:41945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa116e8dce024f9ea6720650d7849d96: Network error: Client connection negotiation failed: client connection to 127.30.255.129:41945: connect: Connection refused (error 111)
I20260812 11:18:50.121299 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 27869
I20260812 11:18:50.130282 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 27129
I20260812 11:18:50.142387 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--webserver_interface=127.30.255.190
--webserver_port=44951
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:50.249179 28207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:50.249408 28207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:50.249461 28207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:50.253278 28207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:18:50.253324 28207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:50.253350 28207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:50.253376 28207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:18:50.253404 28207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:18:50.258374 28207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=44951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:50.259332 28207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:50.260371 28207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:50.268079 28216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.268090 28214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.268115 28220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:50.268573 28207 server_base.cc:1034] running on GCE node
I20260812 11:18:50.269106 28207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:50.270092 28207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:50.271245 28207 hybrid_clock.cc:648] HybridClock initialized: now 1786533530271221 us; error 37 us; skew 500 ppm
Aug 12 11:18:50 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:50.273869 28207 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:18:50.274668 28207 webserver.cc:460] Webserver started at http://127.30.255.190:44951/ using document root <none> and password file <none>
I20260812 11:18:50.275178 28207 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:50.275246 28207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:50.278855 28207 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:18:50.280937 28240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.282049 28207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:18:50.282163 28207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0e7f92da31c7480e97fa18607170b0ab"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "63a06d4a3a967f48afe9af17cff0c8a3"
server_key_iv: "51f5092d6f96fdd0e6ccff2ceb8acdd0"
server_key_version: "encryptionkey@0"
I20260812 11:18:50.282660 28207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:50.312613 28207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:50.315640 28207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:50.315829 28207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:50.322582 28207 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41071
I20260812 11:18:50.322829 28353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41071 every 8 connection(s)
I20260812 11:18:50.323638 28207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:18:50.327653 28355 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:18:50.328150 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 28207
I20260812 11:18:50.328634 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:41945
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=45157
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:50.329633 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap starting.
I20260812 11:18:50.346746 28355 log.cc:824] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:50.357607 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:18:50.357930 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab: Bootstrap complete.
I20260812 11:18:50.361691 28355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:50.362071 28355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Initialized, Role: FOLLOWER
I20260812 11:18:50.362527 28355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:50.362635 28355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:18:50.362692 28355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:18:50.362778 28355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:18:50.364621 28355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:50.364920 28355 leader_election.cc:302] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7f92da31c7480e97fa18607170b0ab; no voters:
I20260812 11:18:50.365200 28355 leader_election.cc:288] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 11:18:50.365296 28367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 3 FOLLOWER]: Leader election won for term 3
I20260812 11:18:50.365487 28367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [term 3 LEADER]: Becoming Leader. State: Replica: 0e7f92da31c7480e97fa18607170b0ab, State: Running, Role: LEADER
I20260812 11:18:50.365777 28367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } }
I20260812 11:18:50.366376 28355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:18:50.366689 28367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e7f92da31c7480e97fa18607170b0ab. Latest consensus state: current_term: 3 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:50.366900 28367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:50.368018 28373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:18:50.368180 28368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e7f92da31c7480e97fa18607170b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7f92da31c7480e97fa18607170b0ab" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41071 } } }
I20260812 11:18:50.368345 28368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7f92da31c7480e97fa18607170b0ab [sys.catalog]: This master's current role is: LEADER
I20260812 11:18:50.372771 28373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:18:50.373410 28373 catalog_manager.cc:1259] Loaded cluster ID: afffc2a418fe45ddafc77f0c3a2492fa
I20260812 11:18:50.373468 28373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 11:18:50.374389 28373 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7efe6bc029ad google::LogMessage::Fail() at ??:0
@ 0x7efe6bc0588a google::LogMessage::SendToLog() at ??:0
@ 0x7efe6bc024de google::LogMessage::Flush() at ??:0
@ 0x7efe6bc03a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7efe76541fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7efe76542d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7efe7653f579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7efe76583640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7efe6de543d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7efe6de54c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7efe6de563f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7efe6de45512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7efe6f4896db start_thread at ??:0
@ 0x7efe6a70061f clone at ??:0
W20260812 11:18:50.462638 28360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:50.462848 28360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:50.462890 28360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:50.466795 28360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:50.466842 28360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:50.466898 28360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:18:50.471877 28360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:41945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=45157
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:50.472779 28360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:50.473719 28360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:50.482775 28435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.482784 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.482790 28442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:50.483208 28360 server_base.cc:1034] running on GCE node
I20260812 11:18:50.483659 28360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:50.484438 28360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:50.485648 28360 hybrid_clock.cc:648] HybridClock initialized: now 1786533530485623 us; error 34 us; skew 500 ppm
Aug 12 11:18:50 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:50.488991 28360 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:18:50.490054 28360 webserver.cc:460] Webserver started at http://127.30.255.129:45157/ using document root <none> and password file <none>
I20260812 11:18:50.490797 28360 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:50.490901 28360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:50.496178 28360 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:18:50.498764 28456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.500254 28360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:18:50.500381 28360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "d71bbfd470b64cdda094f38a14289662"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "55b4a459a0dcb627292c03538019618d"
server_key_iv: "8df9844ae5e0be7eafb18772bbd5b5cf"
server_key_version: "encryptionkey@0"
I20260812 11:18:50.501075 28360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:50.520670 28360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:50.524113 28360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:50.524302 28360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:50.524993 28360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:50.526264 28471 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:18:50.530161 28360 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:18:50.530236 28360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260812 11:18:50.530297 28360 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:18:50.532568 28360 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:18:50.532630 28360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 11:18:50.532869 28471 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Bootstrap starting.
I20260812 11:18:50.543249 28360 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:41945
I20260812 11:18:50.543355 28723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:41945 every 8 connection(s)
I20260812 11:18:50.544582 28360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:18:50.546363 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 28360
I20260812 11:18:50.546835 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:37791
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=35789
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:18:50.551685 28471 log.cc:824] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Log is configured to *not* fsync() on all Append() calls
I20260812 11:18:50.558597 28471 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:18:50.558909 28471 tablet_bootstrap.cc:492] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Bootstrap complete.
I20260812 11:18:50.559888 28471 ts_tablet_manager.cc:1397] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.011s
I20260812 11:18:50.562109 28471 raft_consensus.cc:348] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:50.562485 28471 raft_consensus.cc:729] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71bbfd470b64cdda094f38a14289662, State: Initialized, Role: FOLLOWER
I20260812 11:18:50.562960 28471 consensus_queue.cc:260] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:50.563817 28471 ts_tablet_manager.cc:1428] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
W20260812 11:18:50.617816 28728 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:18:50.546849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:39771)
0812 11:18:50.547422 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.547474 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:50.548446 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:50.617120 (+ 68674us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":174,"threads_started":1}
W20260812 11:18:50.618382 28725 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:41071 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:41071: Client connection negotiation failed: client connection to 127.30.255.190:41071: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:18:50.619120 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.618854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:39847)
0812 11:18:50.619001 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.619064 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 11:18:50.619773 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.619592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:60153)
0812 11:18:50.619683 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.619723 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 11:18:50.619947 28725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:18:50.678457 28734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:50.678681 28734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:50.678721 28734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:50.682802 28734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:50.682857 28734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:50.682910 28734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:18:50.687999 28734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:37791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=35789
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:50.688889 28734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:50.689779 28734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:50.699084 28762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.699146 28756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:50.699158 28734 server_base.cc:1034] running on GCE node
W20260812 11:18:50.699097 28755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:50.699800 28734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:50.700389 28734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:50.701551 28734 hybrid_clock.cc:648] HybridClock initialized: now 1786533530701530 us; error 34 us; skew 500 ppm
Aug 12 11:18:50 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:50.704099 28734 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:18:50.704953 28734 webserver.cc:460] Webserver started at http://127.30.255.130:35789/ using document root <none> and password file <none>
I20260812 11:18:50.705471 28734 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:50.705536 28734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:50.709501 28734 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 11:18:50.711434 28779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.712663 28734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:18:50.712778 28734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "25a84145090641418c9fca266ba9e8d8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "37da1c26f2cf110590e7fd569a1d1c17"
server_key_iv: "cce5c57db5d59d4c82dbf5e80486553f"
server_key_version: "encryptionkey@0"
I20260812 11:18:50.713307 28734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:50.724977 28734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:50.727471 28734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:50.727626 28734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:50.728145 28734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:50.728889 28734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:50.728936 28734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.728986 28734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:50.729012 28734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.737505 28734 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37791
I20260812 11:18:50.737532 29022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37791 every 8 connection(s)
I20260812 11:18:50.738384 28734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:18:50.741814 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.740309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:39903)
0812 11:18:50.740720 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.741165 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
W20260812 11:18:50.742229 29023 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:41071 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:41071: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
I20260812 11:18:50.742758 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.742569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:35803)
0812 11:18:50.742661 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.742708 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 11:18:50.743324 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.743108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:48535)
0812 11:18:50.743197 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.743240 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 11:18:50.743479 29023 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:18:50.745126 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 28734
I20260812 11:18:50.745478 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:43407
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:41071
--webserver_port=38277
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:18:50.853727 29028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:18:50.853957 29028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:18:50.854014 29028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:18:50.857898 29028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:18:50.857949 29028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:18:50.858021 29028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:18:50.862931 29028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:43407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=38277
--tserver_master_addrs=127.30.255.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:18:50.863843 29028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:18:50.864733 29028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:18:50.873432 29045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.873523 29046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:18:50.873446 29049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:18:50.873502 29028 server_base.cc:1034] running on GCE node
I20260812 11:18:50.874025 29028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:18:50.874581 29028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:18:50.875735 29028 hybrid_clock.cc:648] HybridClock initialized: now 1786533530875714 us; error 31 us; skew 500 ppm
Aug 12 11:18:50 dist-test-slave-gjrz krb5kdc[26057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:18:50.878173 29028 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:18:50.878942 29028 webserver.cc:460] Webserver started at http://127.30.255.131:38277/ using document root <none> and password file <none>
I20260812 11:18:50.879464 29028 fs_manager.cc:359] Metadata directory not provided
I20260812 11:18:50.879526 29028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:18:50.883034 29028 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:18:50.884931 29061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.886037 29028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 11:18:50.886138 29028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "f1af60e8d1634391ab2fb73c4cd9aab8"
format_stamp: "Formatted at 2026-08-12 11:18:48 on dist-test-slave-gjrz"
server_key: "a40fd19f295c2fe7b6bd014192698e09"
server_key_iv: "b26e888c7b9438b390b8b20ff0d3fc47"
server_key_version: "encryptionkey@0"
I20260812 11:18:50.886605 29028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:18:50.902257 29028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:18:50.905071 29028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:18:50.905260 29028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:18:50.905771 29028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:18:50.906572 29028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:18:50.906618 29028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.906654 29028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:18:50.906674 29028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:18:50.914865 29028 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:43407
I20260812 11:18:50.914896 29331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:43407 every 8 connection(s)
I20260812 11:18:50.915839 29028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:18:50.919426 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.917875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:52357)
0812 11:18:50.918326 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.918719 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
W20260812 11:18:50.919842 29333 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:41071 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:41071: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
I20260812 11:18:50.920457 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.920221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:49051)
0812 11:18:50.920343 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.920392 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 11:18:50.921017 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:50.920827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:34921)
0812 11:18:50.920924 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:50.920970 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 11:18:50.921164 29333 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:18:50.921756 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 29028
I20260812 11:18:51.620859 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.620436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:52715)
0812 11:18:51.620694 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:51.620763 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260812 11:18:51.744328 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.743952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:45879)
0812 11:18:51.744184 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:51.744233 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
I20260812 11:18:51.921972 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.921599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:48481)
0812 11:18:51.921817 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:51.921865 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":64,"threads_started":1}
I20260812 11:18:51.933319 29346 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:51.933480 29346 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:51.934477 29346 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
Aug 12 11:18:51 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.131@KRBTEST.COM
Aug 12 11:18:51 dist-test-slave-gjrz krb5kdc[26057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533530, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.130@KRBTEST.COM
I20260812 11:18:51.951987 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.934677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37791 (local address 127.30.255.129:60399)
0812 11:18:51.934826 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:51.934840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:51.934939 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:51.936594 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:51.936599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:51.937098 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:51.937575 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:51.937591 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.938813 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.938815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:51.939964 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:51.939975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.940632 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:51.940796 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:51.945852 (+ 5056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:51.945872 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:51.947749 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:51.949603 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:51.949606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:51.949608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:51.949918 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:51.950304 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:51.950309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:51.950464 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:51.951046 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:51.951052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:51.951290 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 11:18:51.951459 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:18:51.951778 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":118,"spinlock_wait_cycles":34432}
I20260812 11:18:51.952024 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.934826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:60415 (local address 127.30.255.131:43407)
0812 11:18:51.935150 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:51.935169 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:51.935496 (+ 327us) server_negotiation.cc:407] Connection header received
0812 11:18:51.936136 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:51.936141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:51.936404 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:51.936648 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:51.937733 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.938681 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940067 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.940501 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940665 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:51.947672 (+ 7007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:51.947679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:51.947695 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:51.947728 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:51.949413 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950050 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:51.950055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:51.950059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:51.950212 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950566 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:51.950569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:51.950571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:51.950935 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:51.951051 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:51.951601 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 11:18:51.951849 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":83,"threads_started":1}
I20260812 11:18:51.952054 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.934834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:60399 (local address 127.30.255.130:37791)
0812 11:18:51.935111 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 11:18:51.935137 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 11:18:51.935183 (+ 46us) server_negotiation.cc:407] Connection header received
0812 11:18:51.935909 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:18:51.935915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:18:51.936239 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:18:51.936528 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:18:51.937733 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.938692 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940075 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.940519 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940672 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:51.947930 (+ 7258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:18:51.947935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:18:51.947946 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:18:51.947971 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:18:51.949479 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950051 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:51.950057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:51.950062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:51.950209 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950551 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:18:51.950555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:18:51.950557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:18:51.950936 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:18:51.951054 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:18:51.951612 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 11:18:51.951865 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260812 11:18:51.952037 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:51.934677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:43407 (local address 127.30.255.129:60415)
0812 11:18:51.935387 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:51.935398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:18:51.935497 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:18:51.936607 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:18:51.936610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:18:51.937100 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:18:51.937575 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:51.937591 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.938810 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.938814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:51.939959 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:18:51.939969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:18:51.940614 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:18:51.940618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:18:51.940793 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:18:51.945851 (+ 5058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:18:51.945875 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:18:51.947438 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:18:51.949527 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:51.949532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:51.949547 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:51.949946 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:51.950304 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:18:51.950307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:18:51.950309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:18:51.950470 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:18:51.951046 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:18:51.951051 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:18:51.951290 (+ 239us) client_negotiation.cc:769] Sending connection context
0812 11:18:51.951457 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 11:18:51.951852 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":725,"spinlock_wait_cycles":61440,"thread_start_us":92,"threads_started":1}
W20260812 11:18:51.954739 28462 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
W20260812 11:18:51.954885 28464 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:51.954970 28464 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d71bbfd470b64cdda094f38a14289662; no voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8
I20260812 11:18:51.955211 29346 raft_consensus.cc:2726] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:18:52.621855 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:52.621479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:58749)
0812 11:18:52.621709 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:52.621759 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260812 11:18:52.745314 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:52.744949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:53861)
0812 11:18:52.745185 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:52.745234 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260812 11:18:52.922933 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:52.922566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:45923)
0812 11:18:52.922789 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:52.922849 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260812 11:18:53.622871 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:53.622512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:32807)
0812 11:18:53.622727 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:53.622777 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260812 11:18:53.671885 29363 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:53.672012 29363 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:53.672417 29363 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
W20260812 11:18:53.673172 28462 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
W20260812 11:18:53.673287 28464 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:53.673367 28464 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d71bbfd470b64cdda094f38a14289662; no voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8
I20260812 11:18:53.673516 29363 raft_consensus.cc:2726] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:18:53.746332 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:53.745923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:50879)
0812 11:18:53.746176 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:53.746230 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":71,"threads_started":1}
I20260812 11:18:53.923954 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:53.923567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:55825)
0812 11:18:53.923804 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:53.923856 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":67,"threads_started":1}
I20260812 11:18:54.623955 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:54.623563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:34861)
0812 11:18:54.623805 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:54.623858 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260812 11:18:54.747391 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:54.746981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:33755)
0812 11:18:54.747226 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:54.747301 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260812 11:18:54.924906 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:54.924535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:43261)
0812 11:18:54.924769 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:54.924822 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260812 11:18:55.625212 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:55.624761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:54423)
0812 11:18:55.625059 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:55.625114 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":72,"threads_started":1}
I20260812 11:18:55.645695 29378 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:55.645829 29378 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:55.646207 29378 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
W20260812 11:18:55.646927 28462 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
W20260812 11:18:55.647114 28464 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:55.647197 28464 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d71bbfd470b64cdda094f38a14289662; no voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8
I20260812 11:18:55.647356 29378 raft_consensus.cc:2726] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:18:55.748387 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:55.748003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:39717)
0812 11:18:55.748250 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:55.748303 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260812 11:18:55.925868 29382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:55.925501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:48725)
0812 11:18:55.925733 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:55.925783 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260812 11:18:56.626262 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:56.625854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:40735)
0812 11:18:56.626112 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:56.626167 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260812 11:18:56.749416 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:56.749049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:54851)
0812 11:18:56.749281 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:56.749331 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260812 11:18:56.926820 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:56.926439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:50841)
0812 11:18:56.926681 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:56.926733 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":69,"threads_started":1}
I20260812 11:18:57.627345 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:57.626934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:58479)
0812 11:18:57.627174 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:57.627229 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260812 11:18:57.750532 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:57.750096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:55509)
0812 11:18:57.750378 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:57.750441 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260812 11:18:57.927817 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:57.927402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:45131)
0812 11:18:57.927682 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:57.927733 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":64,"threads_started":1}
I20260812 11:18:58.533151 29396 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:18:58.533280 29396 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:18:58.533663 29396 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
W20260812 11:18:58.534403 28464 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
W20260812 11:18:58.534518 28462 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
I20260812 11:18:58.534603 28462 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d71bbfd470b64cdda094f38a14289662; no voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8
I20260812 11:18:58.534756 29396 raft_consensus.cc:2726] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:18:58.628384 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:58.627969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:37523)
0812 11:18:58.628237 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:58.628290 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260812 11:18:58.751549 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:58.751179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:50439)
0812 11:18:58.751412 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:58.751462 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260812 11:18:58.928797 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:58.928419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:41477)
0812 11:18:58.928653 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:58.928711 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
I20260812 11:18:59.629403 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:59.629009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:35173)
0812 11:18:59.629251 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:59.629305 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260812 11:18:59.752544 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:59.752179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:43793)
0812 11:18:59.752407 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:59.752459 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260812 11:18:59.929811 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:18:59.929423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:57867)
0812 11:18:59.929673 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 11:18:59.929726 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260812 11:19:00.630645 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:00.630190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:49367)
0812 11:19:00.630444 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:00.630503 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260812 11:19:00.753558 29410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:00.753197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:45363)
0812 11:19:00.753423 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:00.753475 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
I20260812 11:19:00.930790 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:00.930430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:43679)
0812 11:19:00.930655 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:00.930708 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260812 11:19:01.631774 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:01.631367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:40051)
0812 11:19:01.631605 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:01.631659 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260812 11:19:01.754526 29420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:01.754178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:43081)
0812 11:19:01.754392 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:01.754441 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
I20260812 11:19:01.932010 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:01.931482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:40559)
0812 11:19:01.931835 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:01.931903 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260812 11:19:02.632755 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:02.632373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:55131)
0812 11:19:02.632606 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:02.632659 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260812 11:19:02.755534 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:02.755117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:49997)
0812 11:19:02.755385 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:02.755438 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260812 11:19:02.933043 29424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:02.932667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:59461)
0812 11:19:02.932903 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:02.932954 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260812 11:19:03.633834 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:03.633427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:59771)
0812 11:19:03.633686 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:03.633739 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260812 11:19:03.756583 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:03.756145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:35535)
0812 11:19:03.756434 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:03.756491 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260812 11:19:03.934043 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:03.933689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:56723)
0812 11:19:03.933911 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:03.933960 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260812 11:19:04.634871 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:04.634481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:34305)
0812 11:19:04.634721 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:04.634776 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
I20260812 11:19:04.757759 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:04.757306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:35227)
0812 11:19:04.757605 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:04.757668 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260812 11:19:04.935194 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:04.934771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:43233)
0812 11:19:04.935042 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:04.935104 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
I20260812 11:19:05.636191 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:05.635788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.129:57825)
0812 11:19:05.636046 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:05.636105 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260812 11:19:05.705470 29447 raft_consensus.cc:482] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:05.705585 29447 raft_consensus.cc:504] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1af60e8d1634391ab2fb73c4cd9aab8" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 43407 } } peers { permanent_uuid: "25a84145090641418c9fca266ba9e8d8" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37791 } } peers { permanent_uuid: "d71bbfd470b64cdda094f38a14289662" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 41945 } }
I20260812 11:19:05.705982 29447 leader_election.cc:288] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407), 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791)
W20260812 11:19:05.706667 28462 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1af60e8d1634391ab2fb73c4cd9aab8 (127.30.255.131:43407): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
W20260812 11:19:05.706766 28464 leader_election.cc:341] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25a84145090641418c9fca266ba9e8d8 (127.30.255.130:37791): Not found: Tablet not found: fa116e8dce024f9ea6720650d7849d96
I20260812 11:19:05.706826 28464 leader_election.cc:302] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d71bbfd470b64cdda094f38a14289662; no voters: 25a84145090641418c9fca266ba9e8d8, f1af60e8d1634391ab2fb73c4cd9aab8
I20260812 11:19:05.706972 29447 raft_consensus.cc:2726] T fa116e8dce024f9ea6720650d7849d96 P d71bbfd470b64cdda094f38a14289662 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:19:05.758863 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:05.758482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.130:37379)
0812 11:19:05.758704 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:05.758754 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":67,"threads_started":1}
I20260812 11:19:05.929360 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 28360
I20260812 11:19:05.940821 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:05.940473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41071 (local address 127.30.255.131:33203)
0812 11:19:05.940687 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:05.940735 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":60,"threads_started":1}
I20260812 11:19:05.946000 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 28734
I20260812 11:19:05.957609 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 29028
2026-08-12T11:19:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19305 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:19:06 dist-test-slave-gjrz krb5kdc[29456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:19:06 dist-test-slave-gjrz krb5kdc[29456](info): set up 2 sockets
Aug 12 11:19:06 dist-test-slave-gjrz krb5kdc[29456](info): commencing operation
krb5kdc: starting...
W20260812 11:19:07.153762 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:19:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:19:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:19:07.267000 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:39547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:07.373242 29482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:07.373468 29482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:07.373520 29482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:07.377578 29482 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:19:07.377626 29482 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:19:07.377647 29482 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:19:07.377663 29482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:19:07.377687 29482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:07.377719 29482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:07.377748 29482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:19:07.377775 29482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:19:07.382723 29482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:39547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:07.383644 29482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:07.384512 29482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:07.392609 29491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:07.392585 29490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:07.392656 29482 server_base.cc:1034] running on GCE node
W20260812 11:19:07.392606 29496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:07.393273 29482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:07.394246 29482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:07.395401 29482 hybrid_clock.cc:648] HybridClock initialized: now 1786533547395388 us; error 23 us; skew 500 ppm
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:07.397833 29482 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:19:07.398628 29482 webserver.cc:460] Webserver started at http://127.30.255.190:41841/ using document root <none> and password file <none>
I20260812 11:19:07.399114 29482 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:07.399178 29482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:07.399387 29482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:19:07.401288 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "7b49ea77b9c64c7b9a598567b773bbbf"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "ff1cc4dda8c47beec41f9f93cbc5bc8d"
server_key_iv: "056e730b51399b116af91fe9ac7a1f87"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.401806 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "7b49ea77b9c64c7b9a598567b773bbbf"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "ff1cc4dda8c47beec41f9f93cbc5bc8d"
server_key_iv: "056e730b51399b116af91fe9ac7a1f87"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.405189 29482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:19:07.407322 29511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.408399 29482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:07.408509 29482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "7b49ea77b9c64c7b9a598567b773bbbf"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "ff1cc4dda8c47beec41f9f93cbc5bc8d"
server_key_iv: "056e730b51399b116af91fe9ac7a1f87"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.408600 29482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:07.435070 29482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:07.437453 29482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:07.437618 29482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:07.443912 29482 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:39547
I20260812 11:19:07.444178 29630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:39547 every 8 connection(s)
I20260812 11:19:07.444758 29482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:19:07.447302 29632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:07.451828 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 29482
I20260812 11:19:07.451970 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:19:07.452263 31742 external_mini_cluster.cc:1428] Setting key d536eef782ee51c4ee35b5b9e1ef96a7
I20260812 11:19:07.452584 29632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap starting.
I20260812 11:19:07.454890 29632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:07.455617 29632 log.cc:824] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:07.457250 29632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: No bootstrap required, opened a new log
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:07.460788 29632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:07.460980 29632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:07.461035 29632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Initialized, Role: FOLLOWER
I20260812 11:19:07.461460 29632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:07.461609 29632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:19:07.461676 29632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:19:07.461768 29632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:07.462581 29632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:07.462880 29632 leader_election.cc:302] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b49ea77b9c64c7b9a598567b773bbbf; no voters:
I20260812 11:19:07.463178 29632 leader_election.cc:288] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:19:07.463250 29639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:19:07.463578 29639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 LEADER]: Becoming Leader. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Running, Role: LEADER
I20260812 11:19:07.463876 29639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:07.464289 29632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:19:07.465051 29639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b49ea77b9c64c7b9a598567b773bbbf. Latest consensus state: current_term: 1 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:07.465441 29641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:07.465538 29641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:07.465955 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:07.453517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.30.255.190:39547)
0812 11:19:07.453937 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:07.453952 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:07.453986 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:19:07.454581 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:07.454597 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:07.454901 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:07.455217 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:07.456053 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.456983 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.457863 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.458318 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.458468 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:07.460983 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:07.460989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:07.461005 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:07.461037 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:07.462684 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.463087 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.463094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.463100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.463245 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.463610 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.463614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.463615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.464092 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:07.464253 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:07.464735 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 11:19:07.465020 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":134,"threads_started":1}
I20260812 11:19:07.465966 29667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:19:07.465234 29639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:07.469769 29667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:19:07.474478 29667 catalog_manager.cc:1347] Generated new cluster ID: 65e4a62df68d4b669c63f9958fcbc54a
I20260812 11:19:07.474535 29667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:19:07.492384 29667 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:19:07.493453 29667 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:19:07.504994 29667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Generated new TSK 0
I20260812 11:19:07.505754 29667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:19:07.539011 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:07.645601 29692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:07.645817 29692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:07.645872 29692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:07.649729 29692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:07.649780 29692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:07.649854 29692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:19:07.654810 29692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:07.655711 29692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:07.656607 29692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:07.665591 29709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:07.665594 29714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:07.665596 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:07.665958 29692 server_base.cc:1034] running on GCE node
I20260812 11:19:07.666360 29692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:07.666961 29692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:07.668128 29692 hybrid_clock.cc:648] HybridClock initialized: now 1786533547668109 us; error 29 us; skew 500 ppm
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:07.670668 29692 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:19:07.671514 29692 webserver.cc:460] Webserver started at http://127.30.255.129:43625/ using document root <none> and password file <none>
I20260812 11:19:07.672019 29692 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:07.672083 29692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:07.672262 29692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:19:07.674120 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "2803456692a18cee5da826b4f56aa9ab"
server_key_iv: "c7f3fb36f6498f60661e92d72d19e098"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.674644 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "2803456692a18cee5da826b4f56aa9ab"
server_key_iv: "c7f3fb36f6498f60661e92d72d19e098"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.678120 29692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:19:07.680395 29729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.681564 29692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:19:07.681674 29692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "2803456692a18cee5da826b4f56aa9ab"
server_key_iv: "c7f3fb36f6498f60661e92d72d19e098"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.681768 29692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:07.700516 29692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:07.703084 29692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:07.703241 29692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:07.703787 29692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:07.704522 29692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:07.704569 29692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.704617 29692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:07.704645 29692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.712642 29692 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:38467
I20260812 11:19:07.712680 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:38467 every 8 connection(s)
I20260812 11:19:07.713441 29692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:19:07.714504 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 29692
I20260812 11:19:07.714646 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:19:07.714931 31742 external_mini_cluster.cc:1428] Setting key 02296f4cb88ba6c477820c9edf408381
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:19:07.727167 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:07.715210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:53981 (local address 127.30.255.190:39547)
0812 11:19:07.715379 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:07.715384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:07.716482 (+ 1098us) server_negotiation.cc:407] Connection header received
0812 11:19:07.717321 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:07.717324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:07.717379 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:07.717452 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:07.718688 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.719234 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.720762 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.721002 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.721074 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:07.723967 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:07.723970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:07.723972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:07.723998 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:07.725274 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.725821 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.725825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.725826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.725913 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.726185 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.726187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.726188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.726380 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:07.726461 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:07.726934 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 11:19:07.727048 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 11:19:07.727840 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:07.715624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:53981)
0812 11:19:07.716334 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:07.716387 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:07.717138 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:07.717619 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:07.717626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:07.717945 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:07.718536 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:07.718555 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.719373 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.719377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:07.720638 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:07.720650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.721149 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.721153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:07.721340 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:07.721873 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:07.721908 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:07.723768 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:07.725395 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:07.725402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:07.725424 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:07.725712 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:07.726002 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:07.726004 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:07.726006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:07.726120 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:07.726458 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:07.726463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:07.726674 (+ 211us) client_negotiation.cc:769] Sending connection context
0812 11:19:07.726821 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 11:19:07.727051 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:19:07.728945 29974 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:07.729157 29974 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:07.729588 29974 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:07.731009 29556 ts_manager.cc:194] Registered new tserver with Master: c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467)
I20260812 11:19:07.732555 29556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:53981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:19:07.741683 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:07.846645 29987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:07.846854 29987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:07.846894 29987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:07.850695 29987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:07.850741 29987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:07.850804 29987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:19:07.855644 29987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:07.856470 29987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:07.857331 29987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:07.866410 29998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:07.866449 29997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:07.866523 29987 server_base.cc:1034] running on GCE node
W20260812 11:19:07.866469 30006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:07.866974 29987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:07.867573 29987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:07.868731 29987 hybrid_clock.cc:648] HybridClock initialized: now 1786533547868717 us; error 27 us; skew 500 ppm
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:07.871183 29987 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:19:07.871971 29987 webserver.cc:460] Webserver started at http://127.30.255.130:35061/ using document root <none> and password file <none>
I20260812 11:19:07.872430 29987 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:07.872483 29987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:07.872643 29987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:19:07.874375 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "30fc9eba927c4aa3a42727a19f655c5e"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "6b1b39c992e624c60783eb1fa8d3d1bf"
server_key_iv: "cc6d3b51726643102ba97b807add6ba0"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.874846 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "30fc9eba927c4aa3a42727a19f655c5e"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "6b1b39c992e624c60783eb1fa8d3d1bf"
server_key_iv: "cc6d3b51726643102ba97b807add6ba0"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.878104 29987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:19:07.880223 30017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.881279 29987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:19:07.881371 29987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "30fc9eba927c4aa3a42727a19f655c5e"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "6b1b39c992e624c60783eb1fa8d3d1bf"
server_key_iv: "cc6d3b51726643102ba97b807add6ba0"
server_key_version: "encryptionkey@0"
I20260812 11:19:07.881464 29987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:07.895917 29987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:07.898609 29987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:07.898766 29987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:07.899350 29987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:07.900112 29987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:07.900156 29987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.900197 29987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:07.900228 29987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:07.908468 29987 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37459
I20260812 11:19:07.908505 30262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37459 every 8 connection(s)
I20260812 11:19:07.909340 29987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:19:07.917054 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 29987
I20260812 11:19:07.917194 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:19:07.917455 31742 external_mini_cluster.cc:1428] Setting key 413113e3b8cc0eec2da9c13582f9fb95
Aug 12 11:19:07 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:07.921983 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:07.910947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:32851 (local address 127.30.255.190:39547)
0812 11:19:07.911068 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:07.911071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:07.912083 (+ 1012us) server_negotiation.cc:407] Connection header received
0812 11:19:07.912950 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:07.912953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:07.913014 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:07.913089 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:07.914203 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.914734 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.915919 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.916146 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.916237 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:07.918661 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:07.918665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:07.918668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:07.918697 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:07.920022 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.920488 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.920492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.920494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.920570 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:07.920929 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:07.920932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:07.920934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:07.921119 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:07.921200 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:07.921718 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 11:19:07.921823 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:19:07.922802 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:07.911293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:32851)
0812 11:19:07.911910 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:07.911949 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:07.912752 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:07.913260 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:07.913267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:07.913603 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:07.914054 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:07.914068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.914853 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.914856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:07.915797 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:07.915807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:07.916283 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:07.916287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:07.916441 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:07.916926 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:07.916942 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:07.918508 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:07.920131 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:07.920136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:07.920150 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:07.920386 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:07.920652 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:07.920655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:07.920657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:07.920829 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:07.921215 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:07.921221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:07.921499 (+ 278us) client_negotiation.cc:769] Sending connection context
0812 11:19:07.921692 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 11:19:07.922026 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":105,"threads_started":1}
I20260812 11:19:07.924050 30263 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:07.924258 30263 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:07.924698 30263 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:07.925709 29556 ts_manager.cc:194] Registered new tserver with Master: 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
I20260812 11:19:07.926445 29556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:32851
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:19:07.944039 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:08.051788 30271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:08.052016 30271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:08.052070 30271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:08.055930 30271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:08.055977 30271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:08.056049 30271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:19:08.061002 30271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:08.061906 30271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:08.062839 30271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:08.071885 30288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:08.071897 30280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:08.071903 30278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:08.072348 30271 server_base.cc:1034] running on GCE node
I20260812 11:19:08.072726 30271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:08.073318 30271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:08.074487 30271 hybrid_clock.cc:648] HybridClock initialized: now 1786533548074466 us; error 28 us; skew 500 ppm
Aug 12 11:19:08 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:08.077183 30271 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:19:08.077936 30271 webserver.cc:460] Webserver started at http://127.30.255.131:45323/ using document root <none> and password file <none>
I20260812 11:19:08.078434 30271 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:08.078496 30271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:08.078670 30271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:19:08.080588 30271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "5a266282e7014f2b92aa7c960ae00763"
format_stamp: "Formatted at 2026-08-12 11:19:08 on dist-test-slave-gjrz"
server_key: "8977f6892358b24d470c39fd4886ec28"
server_key_iv: "d938e5c1b35594e4b391be5c9c16b6cd"
server_key_version: "encryptionkey@0"
I20260812 11:19:08.081107 30271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "5a266282e7014f2b92aa7c960ae00763"
format_stamp: "Formatted at 2026-08-12 11:19:08 on dist-test-slave-gjrz"
server_key: "8977f6892358b24d470c39fd4886ec28"
server_key_iv: "d938e5c1b35594e4b391be5c9c16b6cd"
server_key_version: "encryptionkey@0"
I20260812 11:19:08.084493 30271 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:19:08.086781 30303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:08.087915 30271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:19:08.088029 30271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "5a266282e7014f2b92aa7c960ae00763"
format_stamp: "Formatted at 2026-08-12 11:19:08 on dist-test-slave-gjrz"
server_key: "8977f6892358b24d470c39fd4886ec28"
server_key_iv: "d938e5c1b35594e4b391be5c9c16b6cd"
server_key_version: "encryptionkey@0"
I20260812 11:19:08.088120 30271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:08.097671 30271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:08.100441 30271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:08.100601 30271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:08.101106 30271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:08.101827 30271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:08.101872 30271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:08.101933 30271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:08.101961 30271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:08.109875 30271 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:44331
I20260812 11:19:08.109925 30515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:44331 every 8 connection(s)
I20260812 11:19:08.110637 30271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
Aug 12 11:19:08 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:08.119692 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 30271
I20260812 11:19:08.119797 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:19:08.120033 31742 external_mini_cluster.cc:1428] Setting key a35ddca3097298676d2613d762acc602
I20260812 11:19:08.122478 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.112215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:50525 (local address 127.30.255.190:39547)
0812 11:19:08.112353 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.112358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.113322 (+ 964us) server_negotiation.cc:407] Connection header received
0812 11:19:08.114109 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.114111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.114157 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.114218 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:08.115220 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.115771 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.116790 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.116987 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.117051 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.119138 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:08.119141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:08.119143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:08.119167 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:08.120551 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:08.121046 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:08.121049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:08.121051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:08.121132 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:08.121490 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:08.121492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:08.121493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:08.121666 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:08.121738 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.122270 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.122366 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:19:08.123159 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.112589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:50525)
0812 11:19:08.113189 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.113227 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.113932 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.114354 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.114360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.114684 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:08.115113 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.115126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.115863 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.115866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.116701 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.116710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.117082 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.117085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.117212 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.117666 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:08.117681 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:08.118980 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:08.120641 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:08.120647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:08.120661 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:08.120957 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:08.121223 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:08.121225 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:08.121227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:08.121396 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:08.121765 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:08.121770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:08.122050 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.122204 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.122426 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":110,"threads_started":1}
I20260812 11:19:08.124068 30516 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:08.124248 30516 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:08.124636 30516 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:08.125555 29556 ts_manager.cc:194] Registered new tserver with Master: 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331)
I20260812 11:19:08.126051 29556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:50525
I20260812 11:19:08.133424 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:19:08.141036 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.135200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.30.255.190:39547)
0812 11:19:08.135331 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.135335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.135477 (+ 142us) server_negotiation.cc:407] Connection header received
0812 11:19:08.135566 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.135567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.135604 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.135664 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:08.136228 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.136688 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.137402 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.137590 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.137652 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.138727 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:08.138729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:08.138731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:08.138746 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:08.139848 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:08.140187 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:08.140189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:08.140191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:08.140251 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:08.140427 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:08.140429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:08.140430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:08.140607 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:08.140702 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.140857 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.140936 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:19:08.143447 29556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:19:08.146173 29556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:19:08.159032 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.153581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:44331 (local address 127.0.0.1:38668)
0812 11:19:08.154075 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.154086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.154209 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.155322 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.155325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.155344 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:08.155430 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.155437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156345 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.156348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.157478 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.157489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.158527 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.158531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.158647 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.158683 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.158792 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.158903 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":99,"threads_started":1}
I20260812 11:19:08.159047 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.153876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.30.255.131:44331)
0812 11:19:08.154883 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.154895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.154914 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:19:08.154974 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.154979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.155136 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.155330 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:08.155535 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156225 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.157604 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.158407 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.158515 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.158648 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.158823 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.158911 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":134,"threads_started":1}
I20260812 11:19:08.159246 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.153468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.0.0.1:37388)
0812 11:19:08.154039 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.154095 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.154377 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.155110 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.155116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.155139 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:08.155345 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.155353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156793 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.156796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.157818 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.157827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.158782 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.158785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.158874 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.158895 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.158957 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.159011 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":52,"spinlock_wait_cycles":1920,"thread_start_us":69,"threads_started":1}
I20260812 11:19:08.159376 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.153600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.30.255.129:38467)
0812 11:19:08.154633 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.154645 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.154667 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:19:08.154744 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.154749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.154951 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.155108 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:08.155480 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156648 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.157932 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.158478 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.158595 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.158719 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.159117 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.159217 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":91,"threads_started":1}
I20260812 11:19:08.160493 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.154165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37459 (local address 127.0.0.1:38938)
0812 11:19:08.154427 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.154440 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.154531 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.155148 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.155150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.155171 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:08.155345 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.155353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156850 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.156853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.158260 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.158272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.159850 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.159854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.159940 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.159964 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.160015 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.160092 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":187,"spinlock_wait_cycles":1792,"thread_start_us":82,"threads_started":1}
I20260812 11:19:08.161084 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.154387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.30.255.130:37459)
0812 11:19:08.154768 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.154781 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.154803 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:19:08.154872 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.154876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.155009 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.155482 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:08.155660 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.156729 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.158405 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.159717 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.160751 (+ 1034us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.160866 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.160905 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.160969 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260812 11:19:08.162039 29821 tablet_service.cc:1467] Processing CreateTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a (DEFAULT_TABLE table=test-table [id=46a3a19b5658492598ef94f9e500e791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:08.162039 30380 tablet_service.cc:1467] Processing CreateTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a (DEFAULT_TABLE table=test-table [id=46a3a19b5658492598ef94f9e500e791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:08.163120 29821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:08.163130 30380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:08.163193 30120 tablet_service.cc:1467] Processing CreateTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a (DEFAULT_TABLE table=test-table [id=46a3a19b5658492598ef94f9e500e791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:08.164112 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:08.170866 30558 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Bootstrap starting.
I20260812 11:19:08.170874 30559 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap starting.
I20260812 11:19:08.171680 30557 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Bootstrap starting.
I20260812 11:19:08.172993 30558 tablet_bootstrap.cc:654] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:08.173290 30559 tablet_bootstrap.cc:654] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:08.173703 30558 log.cc:824] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:08.173916 30557 tablet_bootstrap.cc:654] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:08.174134 30559 log.cc:824] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:08.174674 30557 log.cc:824] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:08.175562 30558 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: No bootstrap required, opened a new log
I20260812 11:19:08.175746 30558 ts_tablet_manager.cc:1397] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:19:08.176365 30559 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: No bootstrap required, opened a new log
I20260812 11:19:08.176551 30559 ts_tablet_manager.cc:1397] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:19:08.176908 30557 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: No bootstrap required, opened a new log
I20260812 11:19:08.177080 30557 ts_tablet_manager.cc:1397] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:19:08.178504 30558 raft_consensus.cc:348] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.178730 30558 raft_consensus.cc:374] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:08.178781 30558 raft_consensus.cc:729] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a266282e7014f2b92aa7c960ae00763, State: Initialized, Role: FOLLOWER
I20260812 11:19:08.179152 30558 consensus_queue.cc:260] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.179751 30516 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
I20260812 11:19:08.180258 30558 ts_tablet_manager.cc:1428] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 11:19:08.180207 30559 raft_consensus.cc:348] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.180437 30559 raft_consensus.cc:374] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:08.180482 30559 raft_consensus.cc:729] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a7b7b3d90e40f190bc8325b4388dae, State: Initialized, Role: FOLLOWER
I20260812 11:19:08.180650 30557 raft_consensus.cc:348] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.180896 30557 raft_consensus.cc:374] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:08.180948 30557 raft_consensus.cc:729] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30fc9eba927c4aa3a42727a19f655c5e, State: Initialized, Role: FOLLOWER
I20260812 11:19:08.180878 30559 consensus_queue.cc:260] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.181337 30557 consensus_queue.cc:260] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.181496 29974 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
I20260812 11:19:08.181766 30559 ts_tablet_manager.cc:1428] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:19:08.182411 30263 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
I20260812 11:19:08.182969 30557 ts_tablet_manager.cc:1428] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 11:19:08.214483 29977 tablet.cc:2369] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:19:08.222580 30568 raft_consensus.cc:482] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:08.222733 30568 raft_consensus.cc:504] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.223831 30568 leader_election.cc:288] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
I20260812 11:19:08.227064 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.224181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:49151 (local address 127.30.255.129:38467)
0812 11:19:08.224306 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.224309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.224319 (+ 10us) server_negotiation.cc:407] Connection header received
0812 11:19:08.224357 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.224359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.224395 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.224456 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:08.224719 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.225214 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226185 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.226561 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226664 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.226716 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.226878 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.226943 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:19:08.227064 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.223990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.30.255.131:49151)
0812 11:19:08.224099 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.224113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.224195 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.224487 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.224491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.224508 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:08.224620 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.224625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.225317 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.225321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.226070 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.226079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.226665 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.226773 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.226791 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.226855 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.226937 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260812 11:19:08.227308 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.224121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:42737 (local address 127.30.255.130:37459)
0812 11:19:08.224231 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.224235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.224482 (+ 247us) server_negotiation.cc:407] Connection header received
0812 11:19:08.224595 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.224598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.224640 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.224701 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:08.224937 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.225412 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226368 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.226750 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226848 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.226895 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.227097 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.227169 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:19:08.227674 29875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4eff3a3b4b4c8fa0b449a1c15de89a" candidate_uuid: "5a266282e7014f2b92aa7c960ae00763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" is_pre_election: true
I20260812 11:19:08.227928 29875 raft_consensus.cc:2454] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a266282e7014f2b92aa7c960ae00763 in term 0.
I20260812 11:19:08.228258 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.224067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37459 (local address 127.30.255.131:42737)
0812 11:19:08.224396 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.224407 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:08.224501 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:08.224722 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:08.224725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:08.224740 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:08.224844 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.224849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.225512 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.225515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.226257 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:08.226266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.226855 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.226859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:08.226988 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.227005 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:19:08.227066 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 11:19:08.228125 (+ 1059us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":21,"thread_start_us":109,"threads_started":1}
I20260812 11:19:08.228353 30306 leader_election.cc:302] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a266282e7014f2b92aa7c960ae00763, c3a7b7b3d90e40f190bc8325b4388dae; no voters:
I20260812 11:19:08.228624 30568 raft_consensus.cc:2781] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:19:08.228705 30568 raft_consensus.cc:482] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:19:08.228753 30568 raft_consensus.cc:3037] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:08.228950 30168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4eff3a3b4b4c8fa0b449a1c15de89a" candidate_uuid: "5a266282e7014f2b92aa7c960ae00763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30fc9eba927c4aa3a42727a19f655c5e" is_pre_election: true
I20260812 11:19:08.229168 30168 raft_consensus.cc:2454] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a266282e7014f2b92aa7c960ae00763 in term 0.
I20260812 11:19:08.229846 30568 raft_consensus.cc:504] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.230137 30568 leader_election.cc:288] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [CANDIDATE]: Term 1 election: Requested vote from peers c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
I20260812 11:19:08.230427 29875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4eff3a3b4b4c8fa0b449a1c15de89a" candidate_uuid: "5a266282e7014f2b92aa7c960ae00763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
I20260812 11:19:08.230468 30168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4eff3a3b4b4c8fa0b449a1c15de89a" candidate_uuid: "5a266282e7014f2b92aa7c960ae00763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30fc9eba927c4aa3a42727a19f655c5e"
I20260812 11:19:08.230528 29875 raft_consensus.cc:3037] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:08.230571 30168 raft_consensus.cc:3037] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:08.231498 29875 raft_consensus.cc:2454] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a266282e7014f2b92aa7c960ae00763 in term 1.
I20260812 11:19:08.231549 30168 raft_consensus.cc:2454] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a266282e7014f2b92aa7c960ae00763 in term 1.
I20260812 11:19:08.231760 30306 leader_election.cc:302] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a266282e7014f2b92aa7c960ae00763, c3a7b7b3d90e40f190bc8325b4388dae; no voters:
I20260812 11:19:08.231951 30568 raft_consensus.cc:2781] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:19:08.232146 30568 raft_consensus.cc:686] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 1 LEADER]: Becoming Leader. State: Replica: 5a266282e7014f2b92aa7c960ae00763, State: Running, Role: LEADER
I20260812 11:19:08.232414 30568 consensus_queue.cc:237] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:08.235149 29556 catalog_manager.cc:5495] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "5a266282e7014f2b92aa7c960ae00763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } health_report { overall_health: UNKNOWN } } }
I20260812 11:19:08.254266 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.251029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.30.255.131:44331)
0812 11:19:08.251131 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:08.251136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:08.251152 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:19:08.251337 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:08.251339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:08.251387 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:08.251470 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:19:08.251704 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.252159 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.252968 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:08.253174 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:08.253258 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:08.253553 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:19:08.253907 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:19:08.254005 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:08.254092 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 11:19:08.254140 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:19:08.259124 30168 raft_consensus.cc:1261] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Refusing update from remote peer 5a266282e7014f2b92aa7c960ae00763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:19:08.259131 29875 raft_consensus.cc:1261] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Refusing update from remote peer 5a266282e7014f2b92aa7c960ae00763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:19:08.259738 30568 consensus_queue.cc:1035] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:19:08.259899 30584 consensus_queue.cc:1035] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:19:08.274178 29555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:19:08.274333 29555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:19:08.276527 29555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Sending DeleteTablet for 3 replicas of tablet ab4eff3a3b4b4c8fa0b449a1c15de89a
I20260812 11:19:08.277160 29821 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:08 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:37388
I20260812 11:19:08.277256 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:08 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:38938
I20260812 11:19:08.277398 30380 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:08 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:38668
I20260812 11:19:08.277784 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 29692
I20260812 11:19:08.277810 30615 tablet_replica.cc:330] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: stopping tablet replica
I20260812 11:19:08.286840 30615 raft_consensus.cc:2229] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:19:08.287122 30615 raft_consensus.cc:2258] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:08.290737 30615 ts_tablet_manager.cc:1905] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:19:08.293489 30615 ts_tablet_manager.cc:1918] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:19:08.293584 30615 log.cc:1196] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/ab4eff3a3b4b4c8fa0b449a1c15de89a
I20260812 11:19:08.293895 30615 ts_tablet_manager.cc:1939] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 5a266282e7014f2b92aa7c960ae00763: Deleting consensus metadata
I20260812 11:19:08.295656 29523 catalog_manager.cc:4841] TS 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): tablet ab4eff3a3b4b4c8fa0b449a1c15de89a (table test-table [id=46a3a19b5658492598ef94f9e500e791]) successfully deleted
I20260812 11:19:08.299458 30613 tablet_replica.cc:330] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: stopping tablet replica
I20260812 11:19:08.299794 30613 raft_consensus.cc:2229] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:19:08.300112 30613 raft_consensus.cc:2258] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:08.301338 30613 ts_tablet_manager.cc:1905] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:19:08.303370 30613 ts_tablet_manager.cc:1918] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:19:08.303457 30613 log.cc:1196] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/ab4eff3a3b4b4c8fa0b449a1c15de89a
I20260812 11:19:08.303712 30613 ts_tablet_manager.cc:1939] T ab4eff3a3b4b4c8fa0b449a1c15de89a P 30fc9eba927c4aa3a42727a19f655c5e: Deleting consensus metadata
W20260812 11:19:08.304080 29522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:38467 (error 108)
I20260812 11:19:08.304323 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 29987
I20260812 11:19:08.308648 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.304883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.0.0.1:37390)
0812 11:19:08.308491 (+ 3608us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.308571 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3488}
W20260812 11:19:08.331476 29522 catalog_manager.cc:4584] TS c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38467: connect: Connection refused (error 111)
I20260812 11:19:08.338006 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 30271
I20260812 11:19:08.342660 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:08.338092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37459 (local address 127.0.0.1:38950)
0812 11:19:08.342489 (+ 4397us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:08.342563 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.130:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4303}
I20260812 11:19:08.348699 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 29482
I20260812 11:19:28.360704 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--webserver_interface=127.30.255.190
--webserver_port=41841
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:39547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:28.469321 30617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:28.469533 30617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:28.469574 30617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:28.473449 30617 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:19:28.473490 30617 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:19:28.473505 30617 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:19:28.473517 30617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:19:28.473551 30617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:28.473580 30617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:28.473605 30617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:19:28.473631 30617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:19:28.478571 30617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:39547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=41841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:28.479485 30617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:28.480383 30617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:28.492925 30631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.492949 30636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.492925 30630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:28.492906 30617 server_base.cc:1034] running on GCE node
I20260812 11:19:28.493698 30617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:28.494715 30617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:28.495862 30617 hybrid_clock.cc:648] HybridClock initialized: now 1786533568495843 us; error 29 us; skew 500 ppm
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:28.498410 30617 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:19:28.499228 30617 webserver.cc:460] Webserver started at http://127.30.255.190:41841/ using document root <none> and password file <none>
I20260812 11:19:28.499764 30617 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:28.499842 30617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:28.503427 30617 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 11:19:28.505369 30649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:28.506464 30617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:28.506580 30617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "7b49ea77b9c64c7b9a598567b773bbbf"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "ff1cc4dda8c47beec41f9f93cbc5bc8d"
server_key_iv: "056e730b51399b116af91fe9ac7a1f87"
server_key_version: "encryptionkey@0"
I20260812 11:19:28.507133 30617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:28.534056 30617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:28.536707 30617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:28.536846 30617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:28.543115 30617 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:39547
I20260812 11:19:28.543414 30762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:39547 every 8 connection(s)
I20260812 11:19:28.544124 30617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:19:28.547436 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 30617
I20260812 11:19:28.547943 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:38467
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=43625
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:19:28.548511 30763 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:19:28.550414 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap starting.
I20260812 11:19:28.566926 30763 log.cc:824] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:28.575214 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:19:28.575553 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap complete.
I20260812 11:19:28.579258 30763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:28.579671 30763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Initialized, Role: FOLLOWER
I20260812 11:19:28.580142 30763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:28.580263 30763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:19:28.580334 30763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:19:28.580427 30763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:19:28.582161 30763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:28.582465 30763 leader_election.cc:302] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b49ea77b9c64c7b9a598567b773bbbf; no voters:
I20260812 11:19:28.582751 30763 leader_election.cc:288] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:19:28.582858 30770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:19:28.583102 30770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 LEADER]: Becoming Leader. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Running, Role: LEADER
I20260812 11:19:28.583475 30770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:28.583885 30763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:19:28.584816 30771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:28.584939 30771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:28.585341 30772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b49ea77b9c64c7b9a598567b773bbbf. Latest consensus state: current_term: 2 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:28.585493 30772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:28.585825 30783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:19:28.589413 30783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:19:28.589900 30783 catalog_manager.cc:1259] Loaded cluster ID: 65e4a62df68d4b669c63f9958fcbc54a
I20260812 11:19:28.589949 30783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:19:28.591678 30783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:19:28.592217 30783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Loaded TSK: 0
I20260812 11:19:28.601998 30783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Generated new TSK 1
I20260812 11:19:28.602203 30783 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:19:28.682546 30765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:28.682842 30765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:28.682900 30765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:28.688627 30765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:28.688692 30765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:28.688771 30765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:19:28.695869 30765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:38467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=43625
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:28.696925 30765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:28.698078 30765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:28.708153 30809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.708186 30810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.708228 30817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:28.708565 30765 server_base.cc:1034] running on GCE node
I20260812 11:19:28.708937 30765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:28.709510 30765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:28.710666 30765 hybrid_clock.cc:648] HybridClock initialized: now 1786533568710637 us; error 44 us; skew 500 ppm
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:28.713375 30765 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:19:28.714344 30765 webserver.cc:460] Webserver started at http://127.30.255.129:43625/ using document root <none> and password file <none>
I20260812 11:19:28.714969 30765 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:28.715078 30765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:28.720031 30765 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:19:28.722476 30832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:28.723893 30765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:19:28.724010 30765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "2803456692a18cee5da826b4f56aa9ab"
server_key_iv: "c7f3fb36f6498f60661e92d72d19e098"
server_key_version: "encryptionkey@0"
I20260812 11:19:28.724665 30765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:28.761860 30765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:28.765074 30765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:28.765249 30765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:28.765861 30765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:28.767016 30846 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:19:28.770931 30765 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:19:28.770999 30765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260812 11:19:28.771062 30765 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:19:28.773072 30765 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:19:28.773114 30765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 11:19:28.773406 30846 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap starting.
I20260812 11:19:28.781922 30765 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:38467
I20260812 11:19:28.781997 31047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:38467 every 8 connection(s)
I20260812 11:19:28.782809 30765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:19:28.785261 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 30765
I20260812 11:19:28.785804 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:37459
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=35061
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:19:28.792564 30846 log.cc:824] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:28.800312 30846 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:19:28.800729 30846 tablet_bootstrap.cc:492] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap complete.
I20260812 11:19:28.801697 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.784867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:55703)
0812 11:19:28.785607 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:28.785663 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:28.786706 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:28.788585 (+ 1879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:28.788593 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:28.789130 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:28.789773 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.789791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.791089 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.791092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.792349 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.792361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.793397 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.793399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.793578 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.794200 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:28.794220 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:28.796063 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:28.798420 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.798426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.798440 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.798798 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.799189 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.799192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.799194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.799400 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.800045 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:28.800051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:28.800325 (+ 274us) client_negotiation.cc:769] Sending connection context
0812 11:19:28.800500 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 11:19:28.800807 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":162,"threads_started":1}
I20260812 11:19:28.801985 30846 ts_tablet_manager.cc:1397] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.008s
I20260812 11:19:28.802125 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.784726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:55703 (local address 127.30.255.190:39547)
0812 11:19:28.785068 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:28.785086 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:28.785782 (+ 696us) server_negotiation.cc:407] Connection header received
0812 11:19:28.787218 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:28.787238 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:28.787734 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:28.788113 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:28.789954 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.790921 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.792490 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.793129 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.793224 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.796269 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:28.796275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:28.796291 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:28.796322 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:28.798293 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.798917 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.798923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.798928 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.799086 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.799498 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.799501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.799503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.799929 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:28.800073 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:28.800658 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 11:19:28.801007 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":99,"threads_started":1}
I20260812 11:19:28.804353 31048 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:28.804587 31048 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:28.804265 30846 raft_consensus.cc:348] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:28.804749 30846 raft_consensus.cc:729] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a7b7b3d90e40f190bc8325b4388dae, State: Initialized, Role: FOLLOWER
I20260812 11:19:28.805080 31048 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:28.805220 30846 consensus_queue.cc:260] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } } peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } }
I20260812 11:19:28.806062 30846 ts_tablet_manager.cc:1428] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260812 11:19:28.807717 30676 ts_manager.cc:194] Registered new tserver with Master: c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467)
I20260812 11:19:28.811061 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:55703
I20260812 11:19:28.813527 31048 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.129@KRBTEST.COM
I20260812 11:19:28.820214 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.810552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.30.255.129:38467)
0812 11:19:28.811029 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:28.811044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:28.811387 (+ 343us) server_negotiation.cc:407] Connection header received
0812 11:19:28.811573 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:28.811579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:28.811768 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:28.811902 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:28.812523 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.813270 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.814240 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.814721 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.814820 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.817103 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:28.817111 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:28.817120 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:28.817140 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:28.818450 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.818951 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.818955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.818956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.819038 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.819336 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.819338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.819339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.819553 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:28.819651 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:28.819968 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 11:19:28.820102 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":111,"threads_started":1}
I20260812 11:19:28.820230 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.810462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.0.0.1:36250)
0812 11:19:28.811233 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:28.811301 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:28.811438 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:28.811920 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:28.811925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:28.812302 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:28.812406 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.812412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.813376 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.813380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.814126 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.814135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.814858 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.814861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.814968 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.815472 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:28.815484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:28.816937 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:28.818563 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.818573 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.818578 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.818857 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.819115 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.819118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.819120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.819214 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.819652 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:28.819658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:28.819774 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 11:19:28.819904 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 11:19:28.820076 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":99,"threads_started":1}
I20260812 11:19:28.821277 30929 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4eff3a3b4b4c8fa0b449a1c15de89a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:08 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:36250
I20260812 11:19:28.821987 31070 tablet_replica.cc:330] stopping tablet replica
I20260812 11:19:28.822237 31070 raft_consensus.cc:2229] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:19:28.822439 31070 raft_consensus.cc:2258] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:28.823294 31070 ts_tablet_manager.cc:1905] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:19:28.826562 31070 ts_tablet_manager.cc:1918] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:19:28.826638 31070 log.cc:1196] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal/wals/ab4eff3a3b4b4c8fa0b449a1c15de89a
I20260812 11:19:28.826843 31070 ts_tablet_manager.cc:1939] T ab4eff3a3b4b4c8fa0b449a1c15de89a P c3a7b7b3d90e40f190bc8325b4388dae: Deleting consensus metadata
I20260812 11:19:28.828310 30651 catalog_manager.cc:4841] TS c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467): tablet ab4eff3a3b4b4c8fa0b449a1c15de89a (table test-table [id=46a3a19b5658492598ef94f9e500e791]) successfully deleted
W20260812 11:19:28.901409 31056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:28.901611 31056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:28.901669 31056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:28.905493 31056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:28.905541 31056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:28.905616 31056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:19:28.910462 31056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:37459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=35061
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:28.911322 31056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:28.912181 31056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:28.920712 31077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.920733 31084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:28.920746 31081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:28.920919 31056 server_base.cc:1034] running on GCE node
I20260812 11:19:28.921367 31056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:28.921923 31056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:28.923084 31056 hybrid_clock.cc:648] HybridClock initialized: now 1786533568923067 us; error 37 us; skew 500 ppm
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:28.925559 31056 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:19:28.926309 31056 webserver.cc:460] Webserver started at http://127.30.255.130:35061/ using document root <none> and password file <none>
I20260812 11:19:28.926765 31056 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:28.926829 31056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:28.930292 31056 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:28.932178 31095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:28.933252 31056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:28.933341 31056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "30fc9eba927c4aa3a42727a19f655c5e"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "6b1b39c992e624c60783eb1fa8d3d1bf"
server_key_iv: "cc6d3b51726643102ba97b807add6ba0"
server_key_version: "encryptionkey@0"
I20260812 11:19:28.933820 31056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:28.945648 31056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:28.948439 31056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:28.948583 31056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:28.949069 31056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:28.949774 31056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:28.949818 31056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:28.949853 31056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:28.949873 31056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:28.958086 31056 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37459
I20260812 11:19:28.958127 31302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37459 every 8 connection(s)
I20260812 11:19:28.959005 31056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:19:28.964633 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31056
I20260812 11:19:28.965170 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:44331
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=45323
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:19:28 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533568, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:28.972474 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.960680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:57639 (local address 127.30.255.190:39547)
0812 11:19:28.960818 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:28.960823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:28.961787 (+ 964us) server_negotiation.cc:407] Connection header received
0812 11:19:28.962589 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:28.962592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:28.962635 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:28.962703 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:28.963815 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.964360 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.965944 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.966208 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.966297 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.969027 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:28.969030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:28.969032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:28.969066 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:28.970386 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.970960 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.970962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.970964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.971027 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:28.971422 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:28.971424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:28.971426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:28.971594 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:28.971667 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:28.972248 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 11:19:28.972353 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:19:28.973536 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:28.961033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:57639)
0812 11:19:28.961657 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:28.961694 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:28.962427 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:28.962847 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:28.962853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:28.963192 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:28.963686 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.963700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.964490 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.964494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.965800 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:28.965814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:28.966355 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:28.966359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:28.966551 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:28.967215 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:28.967231 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:28.968836 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:28.970505 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.970512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.970526 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.970860 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.971116 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:28.971119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:28.971121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:28.971330 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:28.971692 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:28.971698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:28.972011 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 11:19:28.972218 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 11:19:28.972541 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":103,"threads_started":1}
I20260812 11:19:28.974654 31306 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:28.974912 31306 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:28.975515 31306 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:28.976626 30676 ts_manager.cc:194] Registered new tserver with Master: 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
I20260812 11:19:28.977391 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:57639
W20260812 11:19:29.076378 31310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:29.076613 31310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:29.076653 31310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:29.080470 31310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:29.080516 31310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:29.080567 31310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:19:29.085716 31310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:44331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=45323
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:29.086725 31310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:29.087690 31310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:29.096779 31319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:29.096834 31321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:29.096778 31317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:29.096993 31310 server_base.cc:1034] running on GCE node
I20260812 11:19:29.097425 31310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:29.098021 31310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:29.099174 31310 hybrid_clock.cc:648] HybridClock initialized: now 1786533569099147 us; error 38 us; skew 500 ppm
Aug 12 11:19:29 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533569, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:29.101776 31310 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:19:29.102572 31310 webserver.cc:460] Webserver started at http://127.30.255.131:45323/ using document root <none> and password file <none>
I20260812 11:19:29.103085 31310 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:29.103166 31310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:29.106774 31310 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:19:29.108714 31332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:29.109799 31310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:19:29.109910 31310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "5a266282e7014f2b92aa7c960ae00763"
format_stamp: "Formatted at 2026-08-12 11:19:08 on dist-test-slave-gjrz"
server_key: "8977f6892358b24d470c39fd4886ec28"
server_key_iv: "d938e5c1b35594e4b391be5c9c16b6cd"
server_key_version: "encryptionkey@0"
I20260812 11:19:29.110421 31310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:29.155638 31310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:29.158737 31310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:29.158914 31310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:29.159523 31310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:29.160326 31310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:29.160373 31310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:29.160425 31310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:29.160455 31310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:29.168851 31310 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:44331
I20260812 11:19:29.168882 31547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:44331 every 8 connection(s)
I20260812 11:19:29.169760 31310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:19:29.172577 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31310
Aug 12 11:19:29 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533569, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:19:29.181367 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.173214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.30.255.190:39547)
0812 11:19:29.173464 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.173468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.173581 (+ 113us) server_negotiation.cc:407] Connection header received
0812 11:19:29.173709 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.173713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.173789 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.173888 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:29.174797 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.175478 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.176541 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.176772 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.176864 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.178367 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:29.178371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:29.178373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:29.178396 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:29.179848 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.180232 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.180236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.180237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.180313 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.180572 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.180575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.180577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.180747 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:29.180952 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.181087 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.181201 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":176,"spinlock_wait_cycles":14336,"thread_start_us":71,"threads_started":1}
I20260812 11:19:29.183017 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.171422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:36349 (local address 127.30.255.190:39547)
0812 11:19:29.171542 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.171546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.172344 (+ 798us) server_negotiation.cc:407] Connection header received
0812 11:19:29.173710 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.173712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.173789 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.173888 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:29.175210 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.175806 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.177200 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.177489 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.177570 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.179817 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:29.179820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:29.179822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:29.179845 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:29.181011 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.181551 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.181554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.181555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.181627 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.181978 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.181982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.181983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.182153 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:29.182237 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.182769 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.182863 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":40,"spinlock_wait_cycles":37632}
I20260812 11:19:29.183815 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.171761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:36349)
0812 11:19:29.172184 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.172224 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.173478 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:29.174096 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:29.174106 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:29.174543 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:29.175078 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.175092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.175920 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.175923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.177074 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.177086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.177635 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.177640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.177850 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.178349 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:29.178364 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:29.179663 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:29.181116 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:29.181122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:29.181135 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:29.181423 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:29.181733 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:29.181736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:29.181738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:29.181871 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:29.182251 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:29.182258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:29.182541 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 11:19:29.182737 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 11:19:29.183047 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260812 11:19:29.184913 31549 heartbeater.cc:347] Connected to a master server at 127.30.255.190:39547
I20260812 11:19:29.185137 31549 heartbeater.cc:464] Registering TS with master...
I20260812 11:19:29.185585 31549 heartbeater.cc:511] Master 127.30.255.190:39547 requested a full tablet report, sending...
I20260812 11:19:29.186548 30676 ts_manager.cc:194] Registered new tserver with Master: 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331)
I20260812 11:19:29.187064 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:36349
I20260812 11:19:29.194473 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:19:29.198110 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.194992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.30.255.190:39547)
0812 11:19:29.195114 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.195118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.195283 (+ 165us) server_negotiation.cc:407] Connection header received
0812 11:19:29.195442 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.195445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.195529 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.195613 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:19:29.195829 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.196294 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.197108 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.197303 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.197384 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.197676 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:19:29.197784 (+ 108us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 11:19:29.197993 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57400: token expired
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 11:19:29.198168 31053 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57400: token expired
W20260812 11:19:29.198457 30531 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.255.190:39547)
I20260812 11:19:29.204403 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.198790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.30.255.190:39547)
0812 11:19:29.198891 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.198894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.198907 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:19:29.198977 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.198979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.199010 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.199071 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:29.199613 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.200068 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.200763 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.200953 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.201013 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.202035 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:29.202038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:29.202039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:29.202058 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:29.203227 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.203589 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.203591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.203593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.203659 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:29.203833 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:29.203835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:29.203836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:29.203971 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:29.204061 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.204203 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.204280 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:19:29.206389 30676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:19:29.209988 30676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:19:29.217332 30929 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d (DEFAULT_TABLE table=test-table [id=0c7d9739012a41bcb9977b66560e1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:29.217984 30929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0a3d572d884934af750fefe3cb1c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:29.220532 30846 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap starting.
I20260812 11:19:29.220727 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.216123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:44331 (local address 127.0.0.1:56514)
0812 11:19:29.216225 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.216239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.216311 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:29.217018 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:29.217021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:29.217040 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:29.217123 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.217130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.218180 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.218183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.219056 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.219065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.220326 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.220330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.220429 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.220468 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 11:19:29.220546 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 11:19:29.220614 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 11:19:29.221269 30846 tablet_bootstrap.cc:654] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:29.221475 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.216250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.30.255.131:44331)
0812 11:19:29.216573 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.216585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.216607 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:19:29.216673 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.216679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.216876 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.217035 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:29.217213 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.218096 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.219681 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.220212 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.220327 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.220458 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.221263 (+ 805us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.221360 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":154,"threads_started":1}
I20260812 11:19:29.222589 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.216353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37459 (local address 127.0.0.1:45584)
0812 11:19:29.216573 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.216587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.216671 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:29.217681 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:29.217684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:29.217703 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:29.217785 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.217790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.218992 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.218995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.220952 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.220962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.222231 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.222235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.222320 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.222341 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:19:29.222408 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:19:29.222485 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260812 11:19:29.223021 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.216756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.30.255.130:37459)
0812 11:19:29.217242 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.217253 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.217277 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:19:29.217350 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.217354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.217540 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.217687 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:29.217877 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.218889 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.221074 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.221906 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.222034 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.222163 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.222795 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.222887 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":117,"threads_started":1}
I20260812 11:19:29.223184 30846 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: No bootstrap required, opened a new log
I20260812 11:19:29.223305 30846 ts_tablet_manager.cc:1397] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:19:29.223677 30846 raft_consensus.cc:348] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.223773 30846 raft_consensus.cc:374] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:29.223809 30846 raft_consensus.cc:729] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a7b7b3d90e40f190bc8325b4388dae, State: Initialized, Role: FOLLOWER
I20260812 11:19:29.223933 30846 consensus_queue.cc:260] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.224236 31429 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d (DEFAULT_TABLE table=test-table [id=0c7d9739012a41bcb9977b66560e1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:29.224743 30846 ts_tablet_manager.cc:1428] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:19:29.225203 31429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0a3d572d884934af750fefe3cb1c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:29.226532 31179 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d (DEFAULT_TABLE table=test-table [id=0c7d9739012a41bcb9977b66560e1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:19:29.227538 31179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0a3d572d884934af750fefe3cb1c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:19:29.233002 31571 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Bootstrap starting.
I20260812 11:19:29.234851 31572 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Bootstrap starting.
I20260812 11:19:29.235412 31571 tablet_bootstrap.cc:654] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:29.236102 31571 log.cc:824] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:29.236966 31572 tablet_bootstrap.cc:654] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Neither blocks nor log segments found. Creating new log.
I20260812 11:19:29.237784 31572 log.cc:824] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:29.238080 31571 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: No bootstrap required, opened a new log
I20260812 11:19:29.238273 31571 ts_tablet_manager.cc:1397] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:19:29.239575 31572 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: No bootstrap required, opened a new log
I20260812 11:19:29.239738 31572 ts_tablet_manager.cc:1397] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:19:29.241175 31571 raft_consensus.cc:348] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.241386 31571 raft_consensus.cc:374] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:29.241451 31571 raft_consensus.cc:729] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a266282e7014f2b92aa7c960ae00763, State: Initialized, Role: FOLLOWER
I20260812 11:19:29.241832 31571 consensus_queue.cc:260] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.242434 31549 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
I20260812 11:19:29.242525 31572 raft_consensus.cc:348] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.242681 31571 ts_tablet_manager.cc:1428] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 11:19:29.242744 31572 raft_consensus.cc:374] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:19:29.242796 31572 raft_consensus.cc:729] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30fc9eba927c4aa3a42727a19f655c5e, State: Initialized, Role: FOLLOWER
I20260812 11:19:29.243167 31572 consensus_queue.cc:260] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.243785 31306 heartbeater.cc:503] Master 127.30.255.190:39547 was elected leader, sending a full tablet report...
I20260812 11:19:29.244024 31572 ts_tablet_manager.cc:1428] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 11:19:29.261336 31585 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:29.261468 31585 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.262426 31585 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467)
I20260812 11:19:29.265964 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.262621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:44331 (local address 127.30.255.130:40689)
0812 11:19:29.262763 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.262776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.262855 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:29.263071 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:29.263073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:29.263091 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:29.263245 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.263252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.264063 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.264066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.264903 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.264912 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.265575 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.265685 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.265712 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:19:29.265775 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:19:29.265840 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 11:19:29.266108 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.262656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:40689 (local address 127.30.255.131:44331)
0812 11:19:29.262755 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.262760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.262886 (+ 126us) server_negotiation.cc:407] Connection header received
0812 11:19:29.262936 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.262939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.262984 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.263055 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:29.263406 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.263957 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265018 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.265457 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265550 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.265587 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.265918 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.265994 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:19:29.266212 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.262738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:40009 (local address 127.30.255.129:38467)
0812 11:19:29.262870 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.262875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.263012 (+ 137us) server_negotiation.cc:407] Connection header received
0812 11:19:29.263164 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.263167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.263228 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.263350 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:19:29.263598 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.264160 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265170 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.265627 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265738 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.265872 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.266010 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.266098 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:19:29.266458 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.262658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.30.255.130:40009)
0812 11:19:29.262935 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.262948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.263065 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:29.263363 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:29.263366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:29.263383 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:19:29.263494 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.263501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.264265 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.264268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.265048 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:29.265058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.265744 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.265748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:29.265849 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.265875 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:19:29.265983 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 11:19:29.266330 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260812 11:19:29.266630 31460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0a3d572d884934af750fefe3cb1c3d" candidate_uuid: "30fc9eba927c4aa3a42727a19f655c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a266282e7014f2b92aa7c960ae00763" is_pre_election: true
I20260812 11:19:29.266873 31460 raft_consensus.cc:2454] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30fc9eba927c4aa3a42727a19f655c5e in term 0.
I20260812 11:19:29.266933 30966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0a3d572d884934af750fefe3cb1c3d" candidate_uuid: "30fc9eba927c4aa3a42727a19f655c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" is_pre_election: true
I20260812 11:19:29.267199 30966 raft_consensus.cc:2454] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30fc9eba927c4aa3a42727a19f655c5e in term 0.
I20260812 11:19:29.267300 31099 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763; no voters:
I20260812 11:19:29.267552 31585 raft_consensus.cc:2781] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:19:29.267632 31585 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:19:29.267663 31585 raft_consensus.cc:3037] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:29.268553 31585 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.268863 31585 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [CANDIDATE]: Term 1 election: Requested vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467)
I20260812 11:19:29.269152 31460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0a3d572d884934af750fefe3cb1c3d" candidate_uuid: "30fc9eba927c4aa3a42727a19f655c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a266282e7014f2b92aa7c960ae00763"
I20260812 11:19:29.269207 30966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0a3d572d884934af750fefe3cb1c3d" candidate_uuid: "30fc9eba927c4aa3a42727a19f655c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
I20260812 11:19:29.269249 31460 raft_consensus.cc:3037] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:29.269330 30966 raft_consensus.cc:3037] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:19:29.270325 31460 raft_consensus.cc:2454] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30fc9eba927c4aa3a42727a19f655c5e in term 1.
I20260812 11:19:29.270515 30966 raft_consensus.cc:2454] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30fc9eba927c4aa3a42727a19f655c5e in term 1.
I20260812 11:19:29.270573 31099 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763; no voters:
I20260812 11:19:29.270727 31585 raft_consensus.cc:2781] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:19:29.270926 31585 raft_consensus.cc:686] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 1 LEADER]: Becoming Leader. State: Replica: 30fc9eba927c4aa3a42727a19f655c5e, State: Running, Role: LEADER
I20260812 11:19:29.271201 31585 consensus_queue.cc:237] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:29.273976 30675 catalog_manager.cc:5495] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130). New cstate: current_term: 1 leader_uuid: "30fc9eba927c4aa3a42727a19f655c5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } health_report { overall_health: UNKNOWN } } }
W20260812 11:19:29.283952 31050 tablet.cc:2369] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:19:29.289503 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:29.286562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.30.255.130:37459)
0812 11:19:29.286668 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:29.286671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:29.286684 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:19:29.286791 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:29.286793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:29.286840 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:29.286917 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:19:29.287096 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.287607 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.288337 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:29.288538 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:29.288599 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:29.288811 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:19:29.289154 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:19:29.289232 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:29.289335 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 11:19:29.289378 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:19:29.294883 30966 raft_consensus.cc:1261] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Refusing update from remote peer 30fc9eba927c4aa3a42727a19f655c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:19:29.294911 31460 raft_consensus.cc:1261] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Refusing update from remote peer 30fc9eba927c4aa3a42727a19f655c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:19:29.295480 31588 consensus_queue.cc:1035] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:19:29.295645 31585 consensus_queue.cc:1035] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:19:29.300835 31597 mvcc.cc:205] Tried to move back new op lower bound from 7317641499825676288 to 7317641499737178112. Current Snapshot: MvccSnapshot[applied={T|T < 7317641499825676288}]
I20260812 11:19:29.303073 31594 mvcc.cc:205] Tried to move back new op lower bound from 7317641499825676288 to 7317641499737178112. Current Snapshot: MvccSnapshot[applied={T|T < 7317641499825676288 or (T in {7317641499825676288})}]
I20260812 11:19:29.305225 31602 mvcc.cc:205] Tried to move back new op lower bound from 7317641499825676288 to 7317641499737178112. Current Snapshot: MvccSnapshot[applied={T|T < 7317641499825676288}]
I20260812 11:19:29.309262 30675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:19:29.309427 30675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:19:29.311637 30675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Sending DeleteTablet for 3 replicas of tablet 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:29.312245 30929 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:29 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:36250
I20260812 11:19:29.312247 31429 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:29 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:56514
I20260812 11:19:29.312417 31070 tablet_replica.cc:330] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: stopping tablet replica
I20260812 11:19:29.312541 31070 raft_consensus.cc:2229] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:19:29.312672 31070 raft_consensus.cc:2258] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:29.312675 31179 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b0a3d572d884934af750fefe3cb1c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:19:29 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:45584
I20260812 11:19:29.312701 31606 tablet_replica.cc:330] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: stopping tablet replica
I20260812 11:19:29.313078 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 30765
I20260812 11:19:29.313098 31606 raft_consensus.cc:2229] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:19:29.316394 31606 raft_consensus.cc:2258] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:29.322899 31606 ts_tablet_manager.cc:1905] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:19:29.325528 31606 ts_tablet_manager.cc:1918] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:19:29.325662 31606 log.cc:1196] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:29.325990 31606 ts_tablet_manager.cc:1939] T 9b0a3d572d884934af750fefe3cb1c3d P 5a266282e7014f2b92aa7c960ae00763: Deleting consensus metadata
I20260812 11:19:29.333840 30652 catalog_manager.cc:4841] TS 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): tablet 9b0a3d572d884934af750fefe3cb1c3d (table test-table [id=0c7d9739012a41bcb9977b66560e1cc6]) successfully deleted
I20260812 11:19:29.335534 31607 tablet_replica.cc:330] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: stopping tablet replica
I20260812 11:19:29.335865 31607 raft_consensus.cc:2229] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:19:29.336246 31607 raft_consensus.cc:2258] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:19:29.337824 31607 ts_tablet_manager.cc:1905] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 11:19:29.339862 30651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:38467 (error 108)
I20260812 11:19:29.340504 31607 ts_tablet_manager.cc:1918] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:19:29.340605 31607 log.cc:1196] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:29.340922 31607 ts_tablet_manager.cc:1939] T 9b0a3d572d884934af750fefe3cb1c3d P 30fc9eba927c4aa3a42727a19f655c5e: Deleting consensus metadata
W20260812 11:19:29.341078 31567 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:19:29.340784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:38467 (local address 127.0.0.1:36254)
0812 11:19:29.340897 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:29.340912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:29.340982 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38467: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 11:19:29.341216 30651 catalog_manager.cc:4584] TS c3a7b7b3d90e40f190bc8325b4388dae (127.30.255.129:38467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b0a3d572d884934af750fefe3cb1c3d: Network error: Client connection negotiation failed: client connection to 127.30.255.129:38467: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 11:19:29.341974 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 31056
I20260812 11:19:29.342012 30652 catalog_manager.cc:4841] TS 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): tablet 9b0a3d572d884934af750fefe3cb1c3d (table test-table [id=0c7d9739012a41bcb9977b66560e1cc6]) successfully deleted
I20260812 11:19:29.359674 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 31310
I20260812 11:19:29.374948 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 30617
I20260812 11:19:49.388579 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--webserver_interface=127.30.255.190
--webserver_port=41841
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:39547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:19:49.496732 31614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:49.497027 31614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:49.497083 31614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:49.501775 31614 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:19:49.501824 31614 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:19:49.501845 31614 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:19:49.501861 31614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:19:49.501897 31614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:49.501931 31614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:49.501959 31614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:19:49.501987 31614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:19:49.507084 31614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:39547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=41841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:49.507987 31614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:49.508908 31614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:49.517164 31623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:49.517165 31625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:49.517215 31614 server_base.cc:1034] running on GCE node
W20260812 11:19:49.517258 31622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:49.517846 31614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:49.518826 31614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:49.519974 31614 hybrid_clock.cc:648] HybridClock initialized: now 1786533589519955 us; error 32 us; skew 500 ppm
Aug 12 11:19:49 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:49.522445 31614 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:19:49.523312 31614 webserver.cc:460] Webserver started at http://127.30.255.190:41841/ using document root <none> and password file <none>
I20260812 11:19:49.523799 31614 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:49.523862 31614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:49.527341 31614 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 11:19:49.529224 31634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:49.530318 31614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:49.530423 31614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "7b49ea77b9c64c7b9a598567b773bbbf"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "ff1cc4dda8c47beec41f9f93cbc5bc8d"
server_key_iv: "056e730b51399b116af91fe9ac7a1f87"
server_key_version: "encryptionkey@0"
I20260812 11:19:49.530890 31614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:49.547859 31614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:49.550608 31614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:49.550765 31614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:49.557343 31614 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:39547
I20260812 11:19:49.557590 31764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:39547 every 8 connection(s)
I20260812 11:19:49.558297 31614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:19:49.562317 31765 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:19:49.563961 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap starting.
I20260812 11:19:49.565202 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31614
I20260812 11:19:49.565683 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:38467
--local_ip_for_outbound_sockets=127.30.255.129
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=43625
--webserver_interface=127.30.255.129
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:19:49.581355 31765 log.cc:824] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:49.592600 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:19:49.592914 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Bootstrap complete.
I20260812 11:19:49.596752 31765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:49.597107 31765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Initialized, Role: FOLLOWER
I20260812 11:19:49.597549 31765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:49.597672 31765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:19:49.597734 31765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:19:49.597824 31765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:19:49.599603 31765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:49.599907 31765 leader_election.cc:302] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b49ea77b9c64c7b9a598567b773bbbf; no voters:
I20260812 11:19:49.600172 31765 leader_election.cc:288] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 11:19:49.600282 31774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 3 FOLLOWER]: Leader election won for term 3
I20260812 11:19:49.600611 31774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [term 3 LEADER]: Becoming Leader. State: Replica: 7b49ea77b9c64c7b9a598567b773bbbf, State: Running, Role: LEADER
I20260812 11:19:49.600908 31774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } }
I20260812 11:19:49.601269 31765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:19:49.602226 31776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:49.602351 31776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:49.603147 31792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:19:49.603503 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b49ea77b9c64c7b9a598567b773bbbf. Latest consensus state: current_term: 3 leader_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b49ea77b9c64c7b9a598567b773bbbf" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39547 } } }
I20260812 11:19:49.603603 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf [sys.catalog]: This master's current role is: LEADER
I20260812 11:19:49.608024 31792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:19:49.608639 31792 catalog_manager.cc:1259] Loaded cluster ID: 65e4a62df68d4b669c63f9958fcbc54a
I20260812 11:19:49.608696 31792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:19:49.611892 31792 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:19:49.612568 31792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Loaded TSK: 0
I20260812 11:19:49.612658 31792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b49ea77b9c64c7b9a598567b773bbbf: Loaded TSK: 1
F20260812 11:19:49.613306 31792 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffa23f7d9ad google::LogMessage::Fail() at ??:0
@ 0x7ffa23f8088a google::LogMessage::SendToLog() at ??:0
@ 0x7ffa23f7d4de google::LogMessage::Flush() at ??:0
@ 0x7ffa23f7ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffa26c4118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ffa26c39f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ffa2e8e27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ffa2e8bc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ffa2e8bd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ffa2e8fedf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ffa2e91d82b std::function<>::operator()() at ??:0
@ 0x7ffa2e8bcb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ffa2e8bdf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffa2e8ba579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ffa2e8fe640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffa261cf3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffa261cfc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ffa261d13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffa261c0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffa278046db start_thread at ??:0
@ 0x7ffa22a7b61f clone at ??:0
W20260812 11:19:49.697661 31771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:49.697896 31771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:49.697938 31771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:49.701985 31771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:49.702025 31771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:49.702080 31771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:19:49.707171 31771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:38467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=43625
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:49.708177 31771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:49.709288 31771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:49.718502 31854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:49.718520 31859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:49.718886 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:49.719094 31771 server_base.cc:1034] running on GCE node
I20260812 11:19:49.719538 31771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:49.720265 31771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:49.721460 31771 hybrid_clock.cc:648] HybridClock initialized: now 1786533589721432 us; error 42 us; skew 500 ppm
Aug 12 11:19:49 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:49.724715 31771 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:19:49.725771 31771 webserver.cc:460] Webserver started at http://127.30.255.129:43625/ using document root <none> and password file <none>
I20260812 11:19:49.726465 31771 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:49.726557 31771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:49.731480 31771 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:19:49.733986 31872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:49.735441 31771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:19:49.735556 31771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "c3a7b7b3d90e40f190bc8325b4388dae"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "2803456692a18cee5da826b4f56aa9ab"
server_key_iv: "c7f3fb36f6498f60661e92d72d19e098"
server_key_version: "encryptionkey@0"
I20260812 11:19:49.736220 31771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:49.753865 31771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:49.757154 31771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:49.757341 31771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:49.758010 31771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:49.759227 31888 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:19:49.763126 31771 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:19:49.763196 31771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260812 11:19:49.763257 31771 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:19:49.765399 31771 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:19:49.765457 31771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 11:19:49.765750 31888 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap starting.
I20260812 11:19:49.775749 31771 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:38467
I20260812 11:19:49.776039 32092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:38467 every 8 connection(s)
I20260812 11:19:49.776841 31771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:19:49.782956 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 31771
I20260812 11:19:49.783494 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:37459
--local_ip_for_outbound_sockets=127.30.255.130
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=35061
--webserver_interface=127.30.255.130
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:19:49.785077 31888 log.cc:824] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Log is configured to *not* fsync() on all Append() calls
I20260812 11:19:49.791533 31888 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:19:49.791838 31888 tablet_bootstrap.cc:492] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Bootstrap complete.
I20260812 11:19:49.792762 31888 ts_tablet_manager.cc:1397] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.000s
I20260812 11:19:49.794938 31888 raft_consensus.cc:348] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:49.795379 31888 raft_consensus.cc:729] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a7b7b3d90e40f190bc8325b4388dae, State: Initialized, Role: FOLLOWER
I20260812 11:19:49.795823 31888 consensus_queue.cc:260] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:49.796666 31888 ts_tablet_manager.cc:1428] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260812 11:19:49.869037 32095 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:19:49.778959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:37611)
0812 11:19:49.779590 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.779658 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:49.780707 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:49.868300 (+ 87593us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":150,"threads_started":1}
W20260812 11:19:49.869527 32093 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:39547 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:39547: Client connection negotiation failed: client connection to 127.30.255.190:39547: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:19:49.870271 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:49.870025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:58589)
0812 11:19:49.870148 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.870215 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 11:19:49.870893 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:49.870700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:39615)
0812 11:19:49.870801 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.870844 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 11:19:49.871052 32093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:19:49.915211 32100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:49.915427 32100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:49.915485 32100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:49.919301 32100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:49.919344 32100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:49.919416 32100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:19:49.924309 32100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:37459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=35061
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:49.925150 32100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:49.925988 32100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:49.934724 32116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:49.934689 32120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:49.934778 32100 server_base.cc:1034] running on GCE node
W20260812 11:19:49.934689 32118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:49.935338 32100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:49.935871 32100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:49.937026 32100 hybrid_clock.cc:648] HybridClock initialized: now 1786533589937010 us; error 30 us; skew 500 ppm
Aug 12 11:19:49 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:49.939524 32100 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:19:49.940266 32100 webserver.cc:460] Webserver started at http://127.30.255.130:35061/ using document root <none> and password file <none>
I20260812 11:19:49.940732 32100 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:49.940814 32100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:49.944363 32100 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:19:49.946194 32133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:49.947345 32100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:19:49.947458 32100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "30fc9eba927c4aa3a42727a19f655c5e"
format_stamp: "Formatted at 2026-08-12 11:19:07 on dist-test-slave-gjrz"
server_key: "6b1b39c992e624c60783eb1fa8d3d1bf"
server_key_iv: "cc6d3b51726643102ba97b807add6ba0"
server_key_version: "encryptionkey@0"
I20260812 11:19:49.947924 32100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:49.968925 32100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:49.971781 32100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:49.971964 32100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:49.972560 32100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:49.973398 32100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:49.973448 32100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:49.973502 32100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:49.973531 32100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:49.981927 32100 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37459
I20260812 11:19:49.981947 32348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37459 every 8 connection(s)
I20260812 11:19:49.982872 32100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:19:49.983829 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 32100
I20260812 11:19:49.984299 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:44331
--local_ip_for_outbound_sockets=127.30.255.131
--tserver_master_addrs=127.30.255.190:39547
--webserver_port=45323
--webserver_interface=127.30.255.131
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:19:49.986666 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:49.985065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:46709)
0812 11:19:49.985818 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.985953 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":127,"threads_started":1}
W20260812 11:19:49.987190 32349 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:39547 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:39547: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
I20260812 11:19:49.987790 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:49.987577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:51175)
0812 11:19:49.987689 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.987738 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:19:49.988381 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:49.988214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:33741)
0812 11:19:49.988299 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:49.988337 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 11:19:49.988528 32349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:19:50.093705 32353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:19:50.093926 32353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:19:50.093981 32353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:19:50.097867 32353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:19:50.097915 32353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:19:50.097970 32353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:19:50.102906 32353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:44331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=45323
--tserver_master_addrs=127.30.255.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:19:50.103845 32353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:19:50.104792 32353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:19:50.113915 32364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:19:50.113935 32368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:50.114048 32353 server_base.cc:1034] running on GCE node
W20260812 11:19:50.113905 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:19:50.114580 32353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:19:50.115152 32353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:19:50.116322 32353 hybrid_clock.cc:648] HybridClock initialized: now 1786533590116306 us; error 28 us; skew 500 ppm
Aug 12 11:19:50 dist-test-slave-gjrz krb5kdc[29456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533590, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:19:50.118853 32353 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:19:50.119709 32353 webserver.cc:460] Webserver started at http://127.30.255.131:45323/ using document root <none> and password file <none>
I20260812 11:19:50.120199 32353 fs_manager.cc:359] Metadata directory not provided
I20260812 11:19:50.120255 32353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:19:50.124058 32353 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:19:50.125967 32386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:50.127137 32353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:19:50.127235 32353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "5a266282e7014f2b92aa7c960ae00763"
format_stamp: "Formatted at 2026-08-12 11:19:08 on dist-test-slave-gjrz"
server_key: "8977f6892358b24d470c39fd4886ec28"
server_key_iv: "d938e5c1b35594e4b391be5c9c16b6cd"
server_key_version: "encryptionkey@0"
I20260812 11:19:50.127781 32353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:19:50.140326 32353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:19:50.144812 32353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:19:50.145010 32353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:19:50.145563 32353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:19:50.146345 32353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:19:50.146401 32353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:50.146452 32353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:19:50.146481 32353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:19:50.154546 32353 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:44331
I20260812 11:19:50.154590 32603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:44331 every 8 connection(s)
I20260812 11:19:50.155520 32353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:19:50.158967 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.157504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:55697)
0812 11:19:50.158179 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.158286 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":121,"threads_started":1}
W20260812 11:19:50.159394 32604 heartbeater.cc:644] Failed to heartbeat to 127.30.255.190:39547 (0 consecutive failures): Network error: Failed to ping master at 127.30.255.190:39547: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
I20260812 11:19:50.159909 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.159706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:34105)
0812 11:19:50.159802 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.159846 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 11:19:50.160403 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.160240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:57291)
0812 11:19:50.160319 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.160357 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 11:19:50.160542 32604 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:19:50.162336 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 32353
I20260812 11:19:50.872135 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.871663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:58211)
0812 11:19:50.871920 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.871984 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260812 11:19:50.940250 32616 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:50.940454 32616 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:50.941536 32616 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
Aug 12 11:19:50 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.131@KRBTEST.COM
Aug 12 11:19:50 dist-test-slave-gjrz krb5kdc[29456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.130@KRBTEST.COM
I20260812 11:19:50.963449 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.941695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:44331 (local address 127.30.255.129:43945)
0812 11:19:50.941806 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.941820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:50.941934 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:50.944206 (+ 2272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:50.944213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:50.944809 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:50.945591 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:50.945612 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.947487 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.947491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:50.948965 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:50.948977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.949928 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:50.950245 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:50.956604 (+ 6359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:50.956630 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:50.958248 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:50.960568 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:50.960574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:50.960590 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:50.961026 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:50.961400 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:50.961405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:50.961599 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:50.962352 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:50.962358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:50.962716 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 11:19:50.962928 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 11:19:50.963237 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":349,"spinlock_wait_cycles":34560}
I20260812 11:19:50.963459 32618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.941965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37459 (local address 127.30.255.129:38347)
0812 11:19:50.942232 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.942246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:19:50.942338 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:19:50.944206 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:19:50.944213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:19:50.944809 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:19:50.945598 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:50.945612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.947560 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.947563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:50.948965 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:19:50.948977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.949924 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:19:50.950220 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:50.956638 (+ 6418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:19:50.956654 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:19:50.958550 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:19:50.960619 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:50.960622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:50.960624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:50.961026 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:50.961400 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:19:50.961405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:19:50.961617 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:19:50.962352 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:19:50.962358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:19:50.962716 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 11:19:50.962928 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 11:19:50.963321 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":318,"spinlock_wait_cycles":85376,"thread_start_us":59,"threads_started":1}
I20260812 11:19:50.963567 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.941902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:43945 (local address 127.30.255.131:44331)
0812 11:19:50.942344 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:50.942361 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:50.942408 (+ 47us) server_negotiation.cc:407] Connection header received
0812 11:19:50.943308 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:50.943318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:50.943776 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:50.944104 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:50.945809 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.947347 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949088 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.949775 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949937 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:50.958504 (+ 8567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:50.958509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:50.958521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:50.958548 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:50.960449 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961136 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:50.961140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:50.961144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:50.961304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961693 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:50.961696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:50.961698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:50.962237 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:50.962367 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:50.963054 (+ 687us) server_negotiation.cc:299] Negotiation successful
0812 11:19:50.963376 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":84,"threads_started":1}
I20260812 11:19:50.964042 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.942155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:38347 (local address 127.30.255.130:37459)
0812 11:19:50.942438 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 11:19:50.942448 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:19:50.942502 (+ 54us) server_negotiation.cc:407] Connection header received
0812 11:19:50.943308 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:19:50.943318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:19:50.943776 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:19:50.944104 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:19:50.945775 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.947427 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949081 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:19:50.949775 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:19:50.949937 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:19:50.958768 (+ 8831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:19:50.958774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:19:50.958790 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:19:50.958824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:19:50.960508 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961129 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:50.961134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:50.961140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:50.961304 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:19:50.961693 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:19:50.961696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:19:50.961698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:19:50.962237 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:19:50.962367 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:19:50.963054 (+ 687us) server_negotiation.cc:299] Negotiation successful
0812 11:19:50.963376 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
W20260812 11:19:50.965958 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
W20260812 11:19:50.966212 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:50.966275 31877 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3a7b7b3d90e40f190bc8325b4388dae; no voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763
I20260812 11:19:50.966540 32616 raft_consensus.cc:2726] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:19:50.989249 32621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:50.988931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:49013)
0812 11:19:50.989128 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:50.989172 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260812 11:19:51.161384 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:51.161009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:53505)
0812 11:19:51.161247 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:51.161296 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260812 11:19:51.873165 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:51.872771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:35669)
0812 11:19:51.873015 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:51.873070 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":69,"threads_started":1}
I20260812 11:19:51.990212 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:51.989795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:45847)
0812 11:19:51.990062 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:51.990116 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260812 11:19:52.162457 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:52.162037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:49481)
0812 11:19:52.162310 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:52.162360 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":72,"threads_started":1}
I20260812 11:19:52.874235 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:52.873813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:36627)
0812 11:19:52.874073 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:52.874137 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
I20260812 11:19:52.991235 32636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:52.990868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:38421)
0812 11:19:52.991097 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:52.991148 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260812 11:19:52.994391 32638 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:52.994503 32638 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:52.994877 32638 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
W20260812 11:19:52.995675 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
W20260812 11:19:52.995785 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:52.995821 31877 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3a7b7b3d90e40f190bc8325b4388dae; no voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763
I20260812 11:19:52.995949 32638 raft_consensus.cc:2726] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:19:53.163461 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:53.163071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:53859)
0812 11:19:53.163324 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:53.163376 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260812 11:19:53.875376 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:53.874917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:43583)
0812 11:19:53.875184 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:53.875240 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260812 11:19:53.992209 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:53.991860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:41629)
0812 11:19:53.992074 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:53.992126 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260812 11:19:54.164454 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:54.164065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:42551)
0812 11:19:54.164316 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:54.164368 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260812 11:19:54.876557 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:54.876135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:55559)
0812 11:19:54.876409 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:54.876463 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":70,"threads_started":1}
I20260812 11:19:54.993212 32646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:54.992833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:42783)
0812 11:19:54.993064 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:54.993127 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260812 11:19:55.165450 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:55.165096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:35485)
0812 11:19:55.165313 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:55.165364 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":63,"threads_started":1}
I20260812 11:19:55.877552 32649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:55.877182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:44939)
0812 11:19:55.877406 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:55.877457 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260812 11:19:55.994199 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:55.993823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:58363)
0812 11:19:55.994048 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:55.994110 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260812 11:19:56.037272 32652 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:56.037390 32652 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:56.037743 32652 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
W20260812 11:19:56.038482 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
W20260812 11:19:56.038635 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:56.038692 31877 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3a7b7b3d90e40f190bc8325b4388dae; no voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763
I20260812 11:19:56.038828 32652 raft_consensus.cc:2726] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:19:56.166491 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:56.166090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:54781)
0812 11:19:56.166351 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:56.166403 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260812 11:19:56.878579 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:56.878192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:45379)
0812 11:19:56.878430 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:56.878483 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260812 11:19:56.995239 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:56.994795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:41825)
0812 11:19:56.995087 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:56.995151 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":80,"threads_started":1}
I20260812 11:19:57.167544 32658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:57.167104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:52869)
0812 11:19:57.167402 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:57.167455 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":69,"threads_started":1}
I20260812 11:19:57.879575 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:57.879193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:45531)
0812 11:19:57.879432 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:57.879480 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":98,"threads_started":1}
I20260812 11:19:57.996342 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:57.995853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:47011)
0812 11:19:57.996161 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:57.996229 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":65,"threads_started":1}
I20260812 11:19:58.168565 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:58.168168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:55055)
0812 11:19:58.168414 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:58.168468 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260812 11:19:58.549911 32664 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:19:58.550072 32664 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:19:58.550463 32664 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
W20260812 11:19:58.551204 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
W20260812 11:19:58.551404 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:19:58.551465 31877 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3a7b7b3d90e40f190bc8325b4388dae; no voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763
I20260812 11:19:58.551628 32664 raft_consensus.cc:2726] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:19:58.880640 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:58.880253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:58175)
0812 11:19:58.880492 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:58.880546 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260812 11:19:58.997802 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:58.997432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:54301)
0812 11:19:58.997663 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:58.997718 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260812 11:19:59.169603 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:59.169220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:32891)
0812 11:19:59.169458 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:59.169511 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260812 11:19:59.881738 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:59.881367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:48317)
0812 11:19:59.881593 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:59.881643 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260812 11:19:59.998852 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:19:59.998472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:47711)
0812 11:19:59.998709 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 11:19:59.998763 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260812 11:20:00.170660 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:00.170291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:35711)
0812 11:20:00.170504 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:00.170563 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":63,"threads_started":1}
I20260812 11:20:00.882812 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:00.882409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:60575)
0812 11:20:00.882662 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:00.882715 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260812 11:20:00.999890 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:00.999497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:39475)
0812 11:20:00.999746 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:00.999801 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":87,"threads_started":1}
I20260812 11:20:01.171698 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:01.171331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:54613)
0812 11:20:01.171552 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:01.171606 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260812 11:20:01.884111 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:01.883642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:40245)
0812 11:20:01.883943 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:01.884011 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":79,"threads_started":1}
I20260812 11:20:02.001155 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:02.000660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:41737)
0812 11:20:02.000983 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:02.001050 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
I20260812 11:20:02.172920 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:02.172474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:52159)
0812 11:20:02.172761 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:02.172826 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260812 11:20:02.885152 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:02.884775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:60019)
0812 11:20:02.885003 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:02.885055 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260812 11:20:03.002246 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:03.001829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:35961)
0812 11:20:03.002102 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:03.002155 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
I20260812 11:20:03.173979 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:03.173583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:60493)
0812 11:20:03.173837 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:03.173890 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260812 11:20:03.886186 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:03.885753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:39777)
0812 11:20:03.886020 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:03.886076 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260812 11:20:04.003232 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:04.002875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:39037)
0812 11:20:04.003093 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:04.003144 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260812 11:20:04.175019 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:04.174642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:51051)
0812 11:20:04.174872 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:04.174926 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260812 11:20:04.556354 32699 raft_consensus.cc:482] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:04.556473 32699 raft_consensus.cc:504] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a266282e7014f2b92aa7c960ae00763" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 44331 } } peers { permanent_uuid: "30fc9eba927c4aa3a42727a19f655c5e" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37459 } } peers { permanent_uuid: "c3a7b7b3d90e40f190bc8325b4388dae" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 38467 } }
I20260812 11:20:04.556821 32699 leader_election.cc:288] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331), 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459)
W20260812 11:20:04.557577 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a266282e7014f2b92aa7c960ae00763 (127.30.255.131:44331): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
W20260812 11:20:04.557739 31877 leader_election.cc:341] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30fc9eba927c4aa3a42727a19f655c5e (127.30.255.130:37459): Not found: Tablet not found: 9b0a3d572d884934af750fefe3cb1c3d
I20260812 11:20:04.557786 31877 leader_election.cc:302] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3a7b7b3d90e40f190bc8325b4388dae; no voters: 30fc9eba927c4aa3a42727a19f655c5e, 5a266282e7014f2b92aa7c960ae00763
I20260812 11:20:04.557961 32699 raft_consensus.cc:2726] T 9b0a3d572d884934af750fefe3cb1c3d P c3a7b7b3d90e40f190bc8325b4388dae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:20:04.887499 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:04.887051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.129:41235)
0812 11:20:04.887346 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:04.887402 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260812 11:20:05.004386 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:05.004007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.130:43925)
0812 11:20:05.004249 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:05.004299 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260812 11:20:05.164773 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 31771
I20260812 11:20:05.177069 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:05.176523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:39547 (local address 127.30.255.131:48319)
0812 11:20:05.176865 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:05.176943 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":68,"threads_started":1}
I20260812 11:20:05.185771 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 32100
I20260812 11:20:05.201818 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 32353
2026-08-12T11:20:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59246 ms)
[----------] 30 tests from SecurityITest (239628 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:05 dist-test-slave-gjrz krb5kdc[32708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:05 dist-test-slave-gjrz krb5kdc[32708](info): set up 2 sockets
Aug 12 11:20:05 dist-test-slave-gjrz krb5kdc[32708](info): commencing operation
krb5kdc: starting...
W20260812 11:20:06.421212 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:06 dist-test-slave-gjrz krb5kdc[32708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:06.533826 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36483
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:06.641443 32730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:06.641669 32730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:06.641723 32730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:06.645594 32730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:06.645645 32730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:06.645671 32730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:06.645694 32730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:06.645721 32730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:06.650727 32730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:06.651623 32730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:06.652519 32730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:06.660379 32746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:06.660430 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:06.660425 32742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:06.660513 32730 server_base.cc:1034] running on GCE node
I20260812 11:20:06.661072 32730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:06.662003 32730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:06.663153 32730 hybrid_clock.cc:648] HybridClock initialized: now 1786533606663140 us; error 24 us; skew 500 ppm
Aug 12 11:20:06 dist-test-slave-gjrz krb5kdc[32708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:06.665607 32730 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:06.666363 32730 webserver.cc:460] Webserver started at http://127.30.255.190:39185/ using document root <none> and password file <none>
I20260812 11:20:06.666838 32730 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:06.666901 32730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:06.667096 32730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:06.668980 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "a220473f74544ad8b780c5d55044ffb2"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "f79967ad24e2713d3367fe79f14a2bae"
server_key_iv: "696d9640b9411f663fad946a53edaa45"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.669495 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "a220473f74544ad8b780c5d55044ffb2"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "f79967ad24e2713d3367fe79f14a2bae"
server_key_iv: "696d9640b9411f663fad946a53edaa45"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.672814 32730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:06.674885 32767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:06.675985 32730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:06.676100 32730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "a220473f74544ad8b780c5d55044ffb2"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "f79967ad24e2713d3367fe79f14a2bae"
server_key_iv: "696d9640b9411f663fad946a53edaa45"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.676186 32730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:06.694229 32730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:06.696668 32730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:06.696825 32730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:06.703004 32730 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:36483
I20260812 11:20:06.703048 398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:36483 every 8 connection(s)
I20260812 11:20:06.703857 32730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:06.706440 400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:06.708997 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 32730
I20260812 11:20:06.709120 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:06.709457 31742 external_mini_cluster.cc:1428] Setting key ddb34d870ec85b17194dd453db600184
I20260812 11:20:06.712587 400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: Bootstrap starting.
I20260812 11:20:06.715065 400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:06.715821 400 log.cc:824] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:06.717682 400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: No bootstrap required, opened a new log
Aug 12 11:20:06 dist-test-slave-gjrz krb5kdc[32708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:06.721254 400 raft_consensus.cc:348] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } }
I20260812 11:20:06.721477 400 raft_consensus.cc:374] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:06.721527 400 raft_consensus.cc:729] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a220473f74544ad8b780c5d55044ffb2, State: Initialized, Role: FOLLOWER
I20260812 11:20:06.721922 400 consensus_queue.cc:260] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } }
I20260812 11:20:06.722045 400 raft_consensus.cc:388] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:06.722108 400 raft_consensus.cc:482] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:06.722213 400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:06.722965 400 raft_consensus.cc:504] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } }
I20260812 11:20:06.723358 400 leader_election.cc:302] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a220473f74544ad8b780c5d55044ffb2; no voters:
I20260812 11:20:06.723644 400 leader_election.cc:288] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:06.723726 409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:06.723891 409 raft_consensus.cc:686] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [term 1 LEADER]: Becoming Leader. State: Replica: a220473f74544ad8b780c5d55044ffb2, State: Running, Role: LEADER
I20260812 11:20:06.724134 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:06.710709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.30.255.190:36483)
0812 11:20:06.711102 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:06.711120 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:06.711164 (+ 44us) server_negotiation.cc:407] Connection header received
0812 11:20:06.711959 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:06.711981 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:06.712365 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:06.712722 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:06.713365 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.714845 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.715742 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.716358 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.716480 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:06.718707 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:06.718714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:06.718730 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:06.718761 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:06.721202 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:06.721639 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:06.721644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:06.721648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:06.721778 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:06.722059 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:06.722062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:06.722064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:06.722489 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:06.722684 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:06.722947 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 11:20:06.723229 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":105,"threads_started":1}
I20260812 11:20:06.724257 409 consensus_queue.cc:237] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } }
I20260812 11:20:06.724851 400 sys_catalog.cc:564] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:06.726063 411 sys_catalog.cc:455] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a220473f74544ad8b780c5d55044ffb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } } }
I20260812 11:20:06.726284 411 sys_catalog.cc:458] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:06.726064 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a220473f74544ad8b780c5d55044ffb2. Latest consensus state: current_term: 1 leader_uuid: "a220473f74544ad8b780c5d55044ffb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a220473f74544ad8b780c5d55044ffb2" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36483 } } }
I20260812 11:20:06.726466 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:06.726744 427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:06.729389 427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:06.734175 427 catalog_manager.cc:1347] Generated new cluster ID: fe9ffb5c7d834506a04f8e338bdf37ca
I20260812 11:20:06.734238 427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:06.741571 427 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:06.742386 427 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:06.746049 427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: Generated new TSK 0
I20260812 11:20:06.746594 427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:20:06.781400 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:06.888509 447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:06.888731 447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:06.888787 447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:06.892665 447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:06.892716 447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:06.892791 447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:06.897899 447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:06.898808 447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:06.899780 447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:06.908785 456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:06.908769 453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:06.908942 447 server_base.cc:1034] running on GCE node
W20260812 11:20:06.908752 454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:06.909444 447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:06.910012 447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:06.911173 447 hybrid_clock.cc:648] HybridClock initialized: now 1786533606911159 us; error 24 us; skew 500 ppm
Aug 12 11:20:06 dist-test-slave-gjrz krb5kdc[32708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:06.913682 447 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:20:06.914469 447 webserver.cc:460] Webserver started at http://127.30.255.129:33201/ using document root <none> and password file <none>
I20260812 11:20:06.914994 447 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:06.915061 447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:06.915254 447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:06.917198 447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "3ed194e1f7754427bc3073a58d651773"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "c6af05cfecd03a5d324b2d766d773644"
server_key_iv: "829e5faf742d3bf5351c409c0ceccfa4"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.917725 447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "3ed194e1f7754427bc3073a58d651773"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "c6af05cfecd03a5d324b2d766d773644"
server_key_iv: "829e5faf742d3bf5351c409c0ceccfa4"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.921242 447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 11:20:06.923565 466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:06.924710 447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:06.924814 447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "3ed194e1f7754427bc3073a58d651773"
format_stamp: "Formatted at 2026-08-12 11:20:06 on dist-test-slave-gjrz"
server_key: "c6af05cfecd03a5d324b2d766d773644"
server_key_iv: "829e5faf742d3bf5351c409c0ceccfa4"
server_key_version: "encryptionkey@0"
I20260812 11:20:06.924906 447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:06.944782 447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:06.947186 447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:06.947366 447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:06.947902 447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:06.948814 447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:06.948863 447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:06.948897 447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:06.948918 447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:06.957057 447 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:36679
I20260812 11:20:06.957086 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:36679 every 8 connection(s)
I20260812 11:20:06.957846 447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
Aug 12 11:20:06 dist-test-slave-gjrz krb5kdc[32708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:06.966914 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 447
I20260812 11:20:06.967031 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:06.967336 31742 external_mini_cluster.cc:1428] Setting key ec852fe5c6fa10771861075c475d1c6e
I20260812 11:20:06.970510 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:06.959463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:37873 (local address 127.30.255.190:36483)
0812 11:20:06.959582 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:06.959585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:06.960421 (+ 836us) server_negotiation.cc:407] Connection header received
0812 11:20:06.961734 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:06.961737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:06.961783 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:06.961855 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:06.963004 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.963589 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.964705 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.964915 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.964979 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:06.967135 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:06.967139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:06.967143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:06.967170 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:06.968599 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:06.969066 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:06.969069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:06.969071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:06.969157 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:06.969476 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:06.969479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:06.969481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:06.969669 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:06.969756 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:06.970226 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 11:20:06.970352 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:06.971298 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:06.959815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36483 (local address 127.30.255.129:37873)
0812 11:20:06.960269 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:06.960318 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:06.961547 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:06.962087 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:06.962094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:06.962421 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:06.962855 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:06.962870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.963704 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.963707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:06.964607 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:06.964618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:06.965017 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:06.965020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:06.965153 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:06.965578 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:06.965593 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:06.966949 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:06.968712 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:06.968718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:06.968732 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:06.968969 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:06.969241 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:06.969245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:06.969247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:06.969380 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:06.969771 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:06.969778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:06.969998 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 11:20:06.970168 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 11:20:06.970464 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260812 11:20:06.972463 692 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36483
I20260812 11:20:06.972676 692 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:06.973104 692 heartbeater.cc:511] Master 127.30.255.190:36483 requested a full tablet report, sending...
I20260812 11:20:06.974488 334 ts_manager.cc:194] Registered new tserver with Master: 3ed194e1f7754427bc3073a58d651773 (127.30.255.129:36679)
I20260812 11:20:06.976003 334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:37873
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:20:06.995034 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:07.101828 703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:07.102056 703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:07.102093 703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:07.105947 703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:07.105995 703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:07.106055 703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:07.110999 703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:07.111858 703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:07.112735 703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:07.121531 715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:07.121564 714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:07.121585 720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:07.121805 703 server_base.cc:1034] running on GCE node
I20260812 11:20:07.122273 703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:07.122821 703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:07.123967 703 hybrid_clock.cc:648] HybridClock initialized: now 1786533607123953 us; error 24 us; skew 500 ppm
Aug 12 11:20:07 dist-test-slave-gjrz krb5kdc[32708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:07.126631 703 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:20:07.127436 703 webserver.cc:460] Webserver started at http://127.30.255.130:39637/ using document root <none> and password file <none>
I20260812 11:20:07.127940 703 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:07.128000 703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:07.128160 703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:07.130084 703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "0ae7b2928b664261a7c795cbcfb88263"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "c56ec6c6641eddc3670cad1ae5b923bb"
server_key_iv: "f6cf37f4e968e07e29e00c252486e002"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.130594 703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "0ae7b2928b664261a7c795cbcfb88263"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "c56ec6c6641eddc3670cad1ae5b923bb"
server_key_iv: "f6cf37f4e968e07e29e00c252486e002"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.133948 703 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 11:20:07.136108 731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.137202 703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:07.137292 703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "0ae7b2928b664261a7c795cbcfb88263"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "c56ec6c6641eddc3670cad1ae5b923bb"
server_key_iv: "f6cf37f4e968e07e29e00c252486e002"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.137367 703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:07.173316 703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:07.176184 703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:07.176326 703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:07.176827 703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:07.177579 703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:07.177624 703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.177661 703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:07.177682 703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.185781 703 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:39973
I20260812 11:20:07.185815 953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:39973 every 8 connection(s)
I20260812 11:20:07.186579 703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:07.190625 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 703
I20260812 11:20:07.190763 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:07.191046 31742 external_mini_cluster.cc:1428] Setting key ef44ecec4e34f7e94d268730cf930991
Aug 12 11:20:07 dist-test-slave-gjrz krb5kdc[32708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:07.200018 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.188220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:43601 (local address 127.30.255.190:36483)
0812 11:20:07.188348 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.188352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.189141 (+ 789us) server_negotiation.cc:407] Connection header received
0812 11:20:07.190408 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.190411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.190470 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.190572 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:07.191876 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.192544 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.193673 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.193894 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.193981 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.196496 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:07.196500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:07.196503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:07.196531 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:07.197894 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.198480 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.198482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.198484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.198555 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.198936 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.198939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.198941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.199136 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:07.199215 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.199802 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.199901 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:20:07.201072 962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.188558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36483 (local address 127.30.255.130:43601)
0812 11:20:07.188996 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.189037 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.190216 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.190706 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.190715 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.191075 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:07.191732 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.191751 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.192668 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.192671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.193549 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.193559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.194029 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.194034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.194227 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.194710 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:07.194725 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:07.196328 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:07.198027 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:07.198033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:07.198048 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:07.198390 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:07.198637 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:07.198640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:07.198642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:07.198817 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:07.199239 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:07.199245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:07.199566 (+ 321us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.199776 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.200102 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:20:07.202101 954 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36483
I20260812 11:20:07.202316 954 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:07.202744 954 heartbeater.cc:511] Master 127.30.255.190:36483 requested a full tablet report, sending...
I20260812 11:20:07.203711 334 ts_manager.cc:194] Registered new tserver with Master: 0ae7b2928b664261a7c795cbcfb88263 (127.30.255.130:39973)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:20:07.204275 334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:43601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:20:07.217443 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:35019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:07.326066 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:07.326300 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:07.326357 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:07.330382 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:07.330440 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:07.330514 966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:07.335600 966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:35019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:07.336547 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:07.337492 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:07.346526 980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:07.346546 973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:07.346668 966 server_base.cc:1034] running on GCE node
W20260812 11:20:07.346596 976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:07.347229 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:07.347839 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:07.349009 966 hybrid_clock.cc:648] HybridClock initialized: now 1786533607348994 us; error 24 us; skew 500 ppm
Aug 12 11:20:07 dist-test-slave-gjrz krb5kdc[32708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:07.351689 966 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:20:07.352492 966 webserver.cc:460] Webserver started at http://127.30.255.131:37263/ using document root <none> and password file <none>
I20260812 11:20:07.353011 966 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:07.353077 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:07.353273 966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:07.355211 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "4fd77d2675114f97a73317d2dbd08b90"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "f405cbacea7bd580185236aedd909eb0"
server_key_iv: "4790fc813d7d0ff1a625c7ad03d59438"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.355786 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "4fd77d2675114f97a73317d2dbd08b90"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "f405cbacea7bd580185236aedd909eb0"
server_key_iv: "4790fc813d7d0ff1a625c7ad03d59438"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.359311 966 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:07.361545 997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.362735 966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:07.362838 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "4fd77d2675114f97a73317d2dbd08b90"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "f405cbacea7bd580185236aedd909eb0"
server_key_iv: "4790fc813d7d0ff1a625c7ad03d59438"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.362926 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:07.382678 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:07.385390 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:07.385567 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:07.386117 966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:07.386909 966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:07.386965 966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.387017 966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:07.387045 966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.395421 966 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:38587
I20260812 11:20:07.395447 1231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:38587 every 8 connection(s)
I20260812 11:20:07.396232 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:20:07.402849 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 966
I20260812 11:20:07.402967 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:07.403290 31742 external_mini_cluster.cc:1428] Setting key de2fe186c051ffaa32781c84f7bab49a
Aug 12 11:20:07 dist-test-slave-gjrz krb5kdc[32708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:07.408684 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.397849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:44957 (local address 127.30.255.190:36483)
0812 11:20:07.398007 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.398012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.398809 (+ 797us) server_negotiation.cc:407] Connection header received
0812 11:20:07.400129 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.400132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.400180 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.400242 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:07.401283 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.401809 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.403011 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.403353 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.403470 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.405921 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:07.405924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:07.405927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:07.405953 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:07.407042 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.407529 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.407532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.407533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.407613 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.407854 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.407856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.407857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.408025 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:07.408097 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.408480 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.408571 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:20:07.409386 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.398213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36483 (local address 127.30.255.131:44957)
0812 11:20:07.398656 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.398706 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.399918 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.400385 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.400391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.400701 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:07.401172 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.401187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.401926 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.401929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.402839 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.402848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.403495 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.403498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.403666 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.404201 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:07.404219 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:07.405759 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:07.407138 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:07.407143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:07.407153 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:07.407440 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:07.407678 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:07.407680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:07.407681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:07.407798 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:07.408099 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:07.408103 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:07.408306 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.408446 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.408683 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260812 11:20:07.410368 1232 heartbeater.cc:347] Connected to a master server at 127.30.255.190:36483
I20260812 11:20:07.410553 1232 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:07.410944 1232 heartbeater.cc:511] Master 127.30.255.190:36483 requested a full tablet report, sending...
I20260812 11:20:07.411904 334 ts_manager.cc:194] Registered new tserver with Master: 4fd77d2675114f97a73317d2dbd08b90 (127.30.255.131:38587)
I20260812 11:20:07.412423 334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:44957
I20260812 11:20:07.416747 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:07.425086 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.418493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55838 (local address 127.30.255.190:36483)
0812 11:20:07.418609 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.418613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.418803 (+ 190us) server_negotiation.cc:407] Connection header received
0812 11:20:07.418916 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.418919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.418958 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.419020 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:07.419715 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.420197 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.421004 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.421229 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.421313 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.422436 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:07.422440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:07.422443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:07.422468 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:07.423541 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.423923 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.423926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.423928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.424005 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:07.424247 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:07.424250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:07.424252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:07.424451 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:07.424568 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.424850 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.424953 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:20:07.427639 334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:07.430500 334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:07.443780 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.438100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36679 (local address 127.0.0.1:42338)
0812 11:20:07.438379 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.438421 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.438532 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.439215 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.439219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.439237 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:07.439373 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.439381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.440724 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.440727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.442140 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.442150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.443249 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.443252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.443377 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.443417 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.443536 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.443648 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260812 11:20:07.443804 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.438251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.30.255.129:36679)
0812 11:20:07.438534 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.438547 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.438569 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:20:07.438661 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.438667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.438871 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.439042 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:07.439511 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.440593 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.442268 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.443112 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.443252 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.443417 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.443564 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.443666 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
I20260812 11:20:07.444130 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.438760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:38587 (local address 127.0.0.1:59400)
0812 11:20:07.439005 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.439018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.439098 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.440562 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.440565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.440586 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:07.440681 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.440687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.441765 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.441769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.442798 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.442808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.443760 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.443764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.443860 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.443886 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.443937 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.443992 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":30,"thread_start_us":98,"threads_started":1}
I20260812 11:20:07.444592 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.438890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.30.255.131:38587)
0812 11:20:07.440143 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.440151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.440169 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:20:07.440220 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.440224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.440396 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.440578 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:07.440774 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.441639 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.442908 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.443634 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.444292 (+ 658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.444391 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.444430 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.444490 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1164,"thread_start_us":113,"threads_started":1}
I20260812 11:20:07.444654 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.439073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:39973 (local address 127.0.0.1:52772)
0812 11:20:07.439837 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.439849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.439937 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.440393 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.440396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.440414 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:07.440503 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.440509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.442051 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.442054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.443174 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.443185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.444277 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.444281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.444373 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.444401 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.444448 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.444514 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260812 11:20:07.445279 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.439415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.30.255.130:39973)
0812 11:20:07.439764 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.439775 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.439933 (+ 158us) server_negotiation.cc:407] Connection header received
0812 11:20:07.440057 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.440062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.440250 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.440380 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:07.440641 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.441899 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.443344 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.444133 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.444882 (+ 749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.445007 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.445058 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.445146 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":153,"threads_started":1}
I20260812 11:20:07.446878 559 tablet_service.cc:1467] Processing CreateTablet for tablet 6a158adabb894311aff34106b780b4bc (DEFAULT_TABLE table=test-table [id=eb14064920544ad0a24995315bb5bc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:07.447808 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a158adabb894311aff34106b780b4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:07.447566 831 tablet_service.cc:1467] Processing CreateTablet for tablet 6a158adabb894311aff34106b780b4bc (DEFAULT_TABLE table=test-table [id=eb14064920544ad0a24995315bb5bc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:07.448385 831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a158adabb894311aff34106b780b4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:07.453631 1270 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: Bootstrap starting.
I20260812 11:20:07.454898 1271 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Bootstrap starting.
I20260812 11:20:07.455984 1270 tablet_bootstrap.cc:654] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:07.455870 1103 tablet_service.cc:1467] Processing CreateTablet for tablet 6a158adabb894311aff34106b780b4bc (DEFAULT_TABLE table=test-table [id=eb14064920544ad0a24995315bb5bc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:07.456857 1103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a158adabb894311aff34106b780b4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:07.457031 1271 tablet_bootstrap.cc:654] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:07.457116 1270 log.cc:824] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:07.458256 1271 log.cc:824] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:07.458894 1270 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: No bootstrap required, opened a new log
I20260812 11:20:07.459089 1270 ts_tablet_manager.cc:1397] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:20:07.460320 1271 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: No bootstrap required, opened a new log
I20260812 11:20:07.460510 1271 ts_tablet_manager.cc:1397] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:20:07.462167 1270 raft_consensus.cc:348] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.462373 1270 raft_consensus.cc:374] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:07.462443 1270 raft_consensus.cc:729] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed194e1f7754427bc3073a58d651773, State: Initialized, Role: FOLLOWER
I20260812 11:20:07.462862 1270 consensus_queue.cc:260] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.463510 692 heartbeater.cc:503] Master 127.30.255.190:36483 was elected leader, sending a full tablet report...
I20260812 11:20:07.463846 1270 ts_tablet_manager.cc:1428] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 11:20:07.464226 1271 raft_consensus.cc:348] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.464534 1271 raft_consensus.cc:374] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:07.464627 1271 raft_consensus.cc:729] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae7b2928b664261a7c795cbcfb88263, State: Initialized, Role: FOLLOWER
I20260812 11:20:07.464697 1277 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Bootstrap starting.
I20260812 11:20:07.465135 1271 consensus_queue.cc:260] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.465854 954 heartbeater.cc:503] Master 127.30.255.190:36483 was elected leader, sending a full tablet report...
I20260812 11:20:07.466342 1271 ts_tablet_manager.cc:1428] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:20:07.467221 1277 tablet_bootstrap.cc:654] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:07.468446 1277 log.cc:824] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:07.470501 1277 tablet_bootstrap.cc:492] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: No bootstrap required, opened a new log
I20260812 11:20:07.470671 1277 ts_tablet_manager.cc:1397] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 11:20:07.473367 1277 raft_consensus.cc:348] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.473552 1277 raft_consensus.cc:374] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:07.473606 1277 raft_consensus.cc:729] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd77d2675114f97a73317d2dbd08b90, State: Initialized, Role: FOLLOWER
I20260812 11:20:07.473940 1277 consensus_queue.cc:260] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.474483 1232 heartbeater.cc:503] Master 127.30.255.190:36483 was elected leader, sending a full tablet report...
I20260812 11:20:07.474655 1277 ts_tablet_manager.cc:1428] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260812 11:20:07.491173 1285 raft_consensus.cc:482] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:07.491343 1285 raft_consensus.cc:504] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.492296 1285 leader_election.cc:288] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ed194e1f7754427bc3073a58d651773 (127.30.255.129:36679), 0ae7b2928b664261a7c795cbcfb88263 (127.30.255.130:39973)
I20260812 11:20:07.497303 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.492936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:51405 (local address 127.30.255.130:39973)
0812 11:20:07.493046 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.493051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.493277 (+ 226us) server_negotiation.cc:407] Connection header received
0812 11:20:07.493468 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.493471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.493544 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.493651 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:07.493929 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.494618 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.496013 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.496598 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.496705 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.496761 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.497085 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.497178 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:07.497330 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.492486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36679 (local address 127.30.255.131:60305)
0812 11:20:07.492619 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.492632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.492709 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.493282 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.493286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.493307 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:07.493440 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.493446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.494461 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.494464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.495831 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.495844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.496818 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.496822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.496989 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.497052 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.497131 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.497204 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"spinlock_wait_cycles":9600}
I20260812 11:20:07.497332 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.492667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:60305 (local address 127.30.255.129:36679)
0812 11:20:07.492877 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.492884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.492900 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:20:07.492982 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.492985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.493058 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.493150 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:07.493567 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.494331 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.495946 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.496550 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.496771 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.496851 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.497151 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.497232 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 11:20:07.497670 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.492728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:39973 (local address 127.30.255.131:51405)
0812 11:20:07.493198 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.493211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:07.493353 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:07.493662 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:07.493664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:07.493682 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:07.493807 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.493813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.494719 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.494722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.495899 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:07.495912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.496823 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.496827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:07.496942 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.496961 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:20:07.497441 (+ 480us) client_negotiation.cc:240] Negotiation successful
0812 11:20:07.497526 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260812 11:20:07.498243 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a158adabb894311aff34106b780b4bc" candidate_uuid: "4fd77d2675114f97a73317d2dbd08b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed194e1f7754427bc3073a58d651773" is_pre_election: true
I20260812 11:20:07.498248 876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a158adabb894311aff34106b780b4bc" candidate_uuid: "4fd77d2675114f97a73317d2dbd08b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae7b2928b664261a7c795cbcfb88263" is_pre_election: true
I20260812 11:20:07.498528 876 raft_consensus.cc:2454] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd77d2675114f97a73317d2dbd08b90 in term 0.
I20260812 11:20:07.498528 594 raft_consensus.cc:2454] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd77d2675114f97a73317d2dbd08b90 in term 0.
I20260812 11:20:07.498977 999 leader_election.cc:302] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae7b2928b664261a7c795cbcfb88263, 4fd77d2675114f97a73317d2dbd08b90; no voters:
I20260812 11:20:07.499213 1285 raft_consensus.cc:2781] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:07.499316 1285 raft_consensus.cc:482] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:07.499368 1285 raft_consensus.cc:3037] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:07.500313 1285 raft_consensus.cc:504] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.500613 1285 leader_election.cc:288] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [CANDIDATE]: Term 1 election: Requested vote from peers 3ed194e1f7754427bc3073a58d651773 (127.30.255.129:36679), 0ae7b2928b664261a7c795cbcfb88263 (127.30.255.130:39973)
I20260812 11:20:07.500980 876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a158adabb894311aff34106b780b4bc" candidate_uuid: "4fd77d2675114f97a73317d2dbd08b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae7b2928b664261a7c795cbcfb88263"
I20260812 11:20:07.501022 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a158adabb894311aff34106b780b4bc" candidate_uuid: "4fd77d2675114f97a73317d2dbd08b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ed194e1f7754427bc3073a58d651773"
I20260812 11:20:07.501081 876 raft_consensus.cc:3037] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:07.501149 594 raft_consensus.cc:3037] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:07.502101 594 raft_consensus.cc:2454] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd77d2675114f97a73317d2dbd08b90 in term 1.
I20260812 11:20:07.502111 876 raft_consensus.cc:2454] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd77d2675114f97a73317d2dbd08b90 in term 1.
I20260812 11:20:07.502381 999 leader_election.cc:302] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae7b2928b664261a7c795cbcfb88263, 4fd77d2675114f97a73317d2dbd08b90; no voters:
I20260812 11:20:07.502532 1285 raft_consensus.cc:2781] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:07.502734 1285 raft_consensus.cc:686] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 1 LEADER]: Becoming Leader. State: Replica: 4fd77d2675114f97a73317d2dbd08b90, State: Running, Role: LEADER
I20260812 11:20:07.503013 1285 consensus_queue.cc:237] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } }
I20260812 11:20:07.505998 332 catalog_manager.cc:5495] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fd77d2675114f97a73317d2dbd08b90 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "4fd77d2675114f97a73317d2dbd08b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fd77d2675114f97a73317d2dbd08b90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 } health_report { overall_health: UNKNOWN } } }
I20260812 11:20:07.520848 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.517817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.30.255.131:38587)
0812 11:20:07.517928 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.517932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.517944 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:07.518085 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.518088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.518139 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.518224 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:07.518433 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.518920 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.519726 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.519925 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.519994 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.520221 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:07.520584 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:07.520665 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.520711 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.520756 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:20:07.526209 594 raft_consensus.cc:1261] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 1 FOLLOWER]: Refusing update from remote peer 4fd77d2675114f97a73317d2dbd08b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:07.526208 876 raft_consensus.cc:1261] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 1 FOLLOWER]: Refusing update from remote peer 4fd77d2675114f97a73317d2dbd08b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:07.526866 1290 consensus_queue.cc:1035] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae7b2928b664261a7c795cbcfb88263" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:07.527019 1285 consensus_queue.cc:1035] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed194e1f7754427bc3073a58d651773" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 36679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:07.533553 1304 mvcc.cc:205] Tried to move back new op lower bound from 7317641656420548608 to 7317641656334950400. Current Snapshot: MvccSnapshot[applied={T|T < 7317641656420548608}]
I20260812 11:20:07.534559 1303 mvcc.cc:205] Tried to move back new op lower bound from 7317641656420548608 to 7317641656334950400. Current Snapshot: MvccSnapshot[applied={T|T < 7317641656420548608}]
I20260812 11:20:07.542404 333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:07.542538 333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:07.544781 333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a220473f74544ad8b780c5d55044ffb2: Sending DeleteTablet for 3 replicas of tablet 6a158adabb894311aff34106b780b4bc
I20260812 11:20:07.545429 559 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a158adabb894311aff34106b780b4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:07 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:42338
I20260812 11:20:07.545511 1103 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a158adabb894311aff34106b780b4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:07 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:59400
I20260812 11:20:07.545590 831 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a158adabb894311aff34106b780b4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:07 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:52772
I20260812 11:20:07.546094 1323 tablet_replica.cc:330] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773: stopping tablet replica
I20260812 11:20:07.546325 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 447
I20260812 11:20:07.546417 1323 raft_consensus.cc:2229] T 6a158adabb894311aff34106b780b4bc P 3ed194e1f7754427bc3073a58d651773 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:07.546963 1327 tablet_replica.cc:330] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: stopping tablet replica
I20260812 11:20:07.547194 1329 tablet_replica.cc:330] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: stopping tablet replica
I20260812 11:20:07.547583 1327 raft_consensus.cc:2229] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:07.547888 1329 raft_consensus.cc:2229] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:20:07.548282 1329 raft_consensus.cc:2258] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:07.548612 1327 raft_consensus.cc:2258] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:07.556902 1327 ts_tablet_manager.cc:1905] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:07.557839 1329 ts_tablet_manager.cc:1905] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:07.560607 1329 ts_tablet_manager.cc:1918] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:07.563221 1327 ts_tablet_manager.cc:1918] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:07.563388 1327 log.cc:1196] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/6a158adabb894311aff34106b780b4bc
I20260812 11:20:07.563531 1329 log.cc:1196] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/6a158adabb894311aff34106b780b4bc
I20260812 11:20:07.563935 1327 ts_tablet_manager.cc:1939] T 6a158adabb894311aff34106b780b4bc P 0ae7b2928b664261a7c795cbcfb88263: Deleting consensus metadata
I20260812 11:20:07.564291 1329 ts_tablet_manager.cc:1939] T 6a158adabb894311aff34106b780b4bc P 4fd77d2675114f97a73317d2dbd08b90: Deleting consensus metadata
I20260812 11:20:07.573093 307 catalog_manager.cc:4841] TS 0ae7b2928b664261a7c795cbcfb88263 (127.30.255.130:39973): tablet 6a158adabb894311aff34106b780b4bc (table test-table [id=eb14064920544ad0a24995315bb5bc14]) successfully deleted
I20260812 11:20:07.573475 307 catalog_manager.cc:4841] TS 4fd77d2675114f97a73317d2dbd08b90 (127.30.255.131:38587): tablet 6a158adabb894311aff34106b780b4bc (table test-table [id=eb14064920544ad0a24995315bb5bc14]) successfully deleted
W20260812 11:20:07.579356 307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:36679 (error 108)
I20260812 11:20:07.579598 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 703
I20260812 11:20:07.590065 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.589747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:36679 (local address 127.0.0.1:42348)
0812 11:20:07.589919 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:07.589983 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:36679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260812 11:20:07.590320 307 catalog_manager.cc:4584] TS 3ed194e1f7754427bc3073a58d651773 (127.30.255.129:36679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a158adabb894311aff34106b780b4bc: Network error: Client connection negotiation failed: client connection to 127.30.255.129:36679: connect: Connection refused (error 111)
I20260812 11:20:07.599542 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 966
I20260812 11:20:07.617384 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 32730
2026-08-12T11:20:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2414 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T11:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:07Z Disabled control of system clock
I20260812 11:20:07.655058 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41185
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:07.762236 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:07.762470 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:07.762517 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:07.766381 1336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:07.766435 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:07.766456 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:07.766476 1336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:07.766498 1336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:07.771086 1336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:07.772023 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:07.773299 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:07.781307 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:07.781450 1336 server_base.cc:1034] running on GCE node
W20260812 11:20:07.781355 1347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:07.781613 1349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:07.782143 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:07.783159 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:07.784318 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786533607784307 us; error 28 us; skew 500 ppm
I20260812 11:20:07.785971 1336 webserver.cc:460] Webserver started at http://127.30.255.190:38537/ using document root <none> and password file <none>
I20260812 11:20:07.786489 1336 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:07.786561 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:07.786752 1336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:07.788702 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "929e9f71a733daffe9cbe22d8af05d91"
server_key_iv: "323ba854e6e779c696b761135a52d4e9"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.789237 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "929e9f71a733daffe9cbe22d8af05d91"
server_key_iv: "323ba854e6e779c696b761135a52d4e9"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.792588 1336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:07.794757 1365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:07.795889 1336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:07.795998 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d"
format_stamp: "Formatted at 2026-08-12 11:20:07 on dist-test-slave-gjrz"
server_key: "929e9f71a733daffe9cbe22d8af05d91"
server_key_iv: "323ba854e6e779c696b761135a52d4e9"
server_key_version: "encryptionkey@0"
I20260812 11:20:07.796084 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:07.808511 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:07.809046 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:07.809198 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:07.815186 1336 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41185
I20260812 11:20:07.815208 1472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41185 every 8 connection(s)
I20260812 11:20:07.815997 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:07.818768 1473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:07.819698 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1336
I20260812 11:20:07.819823 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:07.820056 31742 external_mini_cluster.cc:1428] Setting key b8b4b55b8d19f0d5c3e1c807a0da77bb
I20260812 11:20:07.825008 1473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: Bootstrap starting.
I20260812 11:20:07.827411 1473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:07.828466 1473 log.cc:824] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:07.829722 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:07.821341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.30.255.190:41185)
0812 11:20:07.822001 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:07.822022 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:07.822063 (+ 41us) server_negotiation.cc:407] Connection header received
0812 11:20:07.822876 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:07.822900 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:07.823404 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:07.823773 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:07.824210 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.825592 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.826458 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:07.827104 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:07.827291 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:07.827920 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:07.827927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:07.827941 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:07.828391 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:07.828454 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:07.828481 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:07.828584 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:07.828688 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 11:20:07.828925 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
I20260812 11:20:07.830494 1473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: No bootstrap required, opened a new log
I20260812 11:20:07.833940 1473 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } }
I20260812 11:20:07.834141 1473 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:07.834182 1473 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f72fa14a074d9a9f1e6ca85bbf5d3d, State: Initialized, Role: FOLLOWER
I20260812 11:20:07.834537 1473 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } }
I20260812 11:20:07.834656 1473 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:07.834702 1473 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:07.834769 1473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:07.835561 1473 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } }
I20260812 11:20:07.835842 1473 leader_election.cc:302] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2f72fa14a074d9a9f1e6ca85bbf5d3d; no voters:
I20260812 11:20:07.836066 1473 leader_election.cc:288] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:07.836162 1483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:07.836373 1483 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [term 1 LEADER]: Becoming Leader. State: Replica: b2f72fa14a074d9a9f1e6ca85bbf5d3d, State: Running, Role: LEADER
I20260812 11:20:07.836702 1483 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } }
I20260812 11:20:07.837141 1473 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:07.838116 1487 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } } }
I20260812 11:20:07.838243 1487 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:07.838614 1515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:07.838572 1493 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2f72fa14a074d9a9f1e6ca85bbf5d3d. Latest consensus state: current_term: 1 leader_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f72fa14a074d9a9f1e6ca85bbf5d3d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41185 } } }
I20260812 11:20:07.839154 1493 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:07.841799 1515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:07.846819 1515 catalog_manager.cc:1347] Generated new cluster ID: 355cdaa1fcdc49e7aab0cbdb31d11e36
I20260812 11:20:07.846912 1515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:07.862187 1515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:07.863324 1515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:07.873085 1515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: Generated new TSK 0
I20260812 11:20:07.873746 1515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:07.883744 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--builtin_ntp_servers=127.30.255.148:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:07.992409 1535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:07.992636 1535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:07.992671 1535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:07.996592 1535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:07.996639 1535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:07.996687 1535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:08.001266 1535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:08.002180 1535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:08.003444 1535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:08.012323 1547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:08.012333 1544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:08.012352 1545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:08.012667 1535 server_base.cc:1034] running on GCE node
I20260812 11:20:08.013044 1535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:08.013646 1535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:08.014791 1535 hybrid_clock.cc:648] HybridClock initialized: now 1786533608014767 us; error 36 us; skew 500 ppm
I20260812 11:20:08.016475 1535 webserver.cc:460] Webserver started at http://127.30.255.129:46607/ using document root <none> and password file <none>
I20260812 11:20:08.016994 1535 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:08.017055 1535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:08.017262 1535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:08.019182 1535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "0f60691e6eab4636a718c8ca64d1bef6"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "cd3eee591562b9aecfbe259dd7b815d4"
server_key_iv: "65d1f37a8c7ec3f6f4f51db5b7fdd7fd"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.019768 1535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "0f60691e6eab4636a718c8ca64d1bef6"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "cd3eee591562b9aecfbe259dd7b815d4"
server_key_iv: "65d1f37a8c7ec3f6f4f51db5b7fdd7fd"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.023218 1535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:20:08.025435 1560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.026532 1535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:08.026638 1535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "0f60691e6eab4636a718c8ca64d1bef6"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "cd3eee591562b9aecfbe259dd7b815d4"
server_key_iv: "65d1f37a8c7ec3f6f4f51db5b7fdd7fd"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.026731 1535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:08.049759 1535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:08.050387 1535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:08.050552 1535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:08.051101 1535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:08.052043 1535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:08.052094 1535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.052146 1535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:08.052174 1535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.060199 1535 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:39875
I20260812 11:20:08.060271 1804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:39875 every 8 connection(s)
I20260812 11:20:08.061038 1535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:20:08.069330 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.062696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:47411 (local address 127.30.255.190:41185)
0812 11:20:08.062856 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.062861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.063842 (+ 981us) server_negotiation.cc:407] Connection header received
0812 11:20:08.065176 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.065179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.065228 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.065298 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:08.066118 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.066633 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.067759 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.067980 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.068064 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.068959 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:08.068962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:08.068964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:08.068976 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:08.068993 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:08.069001 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:08.069065 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.069172 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.069220 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:20:08.070087 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.063073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41185 (local address 127.30.255.129:47411)
0812 11:20:08.063703 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.063741 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.064996 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.065442 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.065449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.065530 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:08.066008 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.066022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.066716 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.066719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.067645 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.067656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.068096 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.068098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.068232 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.068711 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:08.068713 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:08.068749 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:08.068809 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:08.069073 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:08.069078 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:08.069082 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.069164 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.069364 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":134,"threads_started":1}
I20260812 11:20:08.070317 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1535
I20260812 11:20:08.070425 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:08.070748 31742 external_mini_cluster.cc:1428] Setting key e714c4733f489384e5940fb7fd923ffe
I20260812 11:20:08.071235 1805 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41185
I20260812 11:20:08.071481 1805 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:08.072016 1805 heartbeater.cc:511] Master 127.30.255.190:41185 requested a full tablet report, sending...
I20260812 11:20:08.072737 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--builtin_ntp_servers=127.30.255.148:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:08.073552 1398 ts_manager.cc:194] Registered new tserver with Master: 0f60691e6eab4636a718c8ca64d1bef6 (127.30.255.129:39875)
I20260812 11:20:08.074842 1398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.129:47411
W20260812 11:20:08.179411 1809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:08.179636 1809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:08.179687 1809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:08.183521 1809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:08.183562 1809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:08.183632 1809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:08.188124 1809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:08.189016 1809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:08.190224 1809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:08.199075 1826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:08.199057 1821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:08.199211 1809 server_base.cc:1034] running on GCE node
W20260812 11:20:08.199335 1822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:08.199880 1809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:08.200481 1809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:08.201648 1809 hybrid_clock.cc:648] HybridClock initialized: now 1786533608201634 us; error 30 us; skew 500 ppm
I20260812 11:20:08.203302 1809 webserver.cc:460] Webserver started at http://127.30.255.130:33675/ using document root <none> and password file <none>
I20260812 11:20:08.203835 1809 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:08.203900 1809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:08.204082 1809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:08.205968 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "d6b331ac799b4bd0a14c797b311329c1"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "3fe6e526c44a839766ce424772fb5ba4"
server_key_iv: "c4347ce32f6d8e1eb1831b3b70e5800c"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.206478 1809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "d6b331ac799b4bd0a14c797b311329c1"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "3fe6e526c44a839766ce424772fb5ba4"
server_key_iv: "c4347ce32f6d8e1eb1831b3b70e5800c"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.209892 1809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:08.212092 1836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.213173 1809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:08.213277 1809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "d6b331ac799b4bd0a14c797b311329c1"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "3fe6e526c44a839766ce424772fb5ba4"
server_key_iv: "c4347ce32f6d8e1eb1831b3b70e5800c"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.213364 1809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:08.226621 1809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:08.227154 1809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:08.227344 1809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:08.227845 1809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:08.228669 1809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:08.228717 1809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.228765 1809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:08.228793 1809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.236557 1809 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:39685
I20260812 11:20:08.236582 2057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:39685 every 8 connection(s)
I20260812 11:20:08.237350 1809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:08.239166 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 1809
I20260812 11:20:08.239339 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:08.239689 31742 external_mini_cluster.cc:1428] Setting key 15cccf0cee60a9bd4ce4686d58d1718e
I20260812 11:20:08.241618 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--builtin_ntp_servers=127.30.255.148:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:08.246162 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.238942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:50237 (local address 127.30.255.190:41185)
0812 11:20:08.239059 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.239063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.240295 (+ 1232us) server_negotiation.cc:407] Connection header received
0812 11:20:08.241705 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.241709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.241766 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.241840 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:08.242752 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.243303 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.244415 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.244651 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.244741 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.245714 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:08.245718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:08.245721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:08.245736 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:08.245755 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:08.245763 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:08.245820 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.245958 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.246020 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:08.246948 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.239416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41185 (local address 127.30.255.130:50237)
0812 11:20:08.240134 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.240188 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.241489 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.242024 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.242031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.242110 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:08.242605 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.242621 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.243430 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.243432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.244291 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.244301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.244789 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.244794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.244947 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.245475 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:08.245476 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:08.245513 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:08.245573 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:08.245832 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:08.245841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:08.245847 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.245949 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.246223 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":146,"threads_started":1}
I20260812 11:20:08.248008 2059 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41185
I20260812 11:20:08.248219 2059 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:08.248680 2059 heartbeater.cc:511] Master 127.30.255.190:41185 requested a full tablet report, sending...
I20260812 11:20:08.249678 1398 ts_manager.cc:194] Registered new tserver with Master: d6b331ac799b4bd0a14c797b311329c1 (127.30.255.130:39685)
I20260812 11:20:08.250416 1398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.130:50237
W20260812 11:20:08.350682 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:08.350900 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:08.350935 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:08.354795 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:08.354839 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:08.354889 2063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:08.359411 2063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:08.360250 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:08.361428 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:08.370108 2072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:08.370111 2076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:08.370163 2073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:08.370321 2063 server_base.cc:1034] running on GCE node
I20260812 11:20:08.370762 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:08.371340 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:08.372493 2063 hybrid_clock.cc:648] HybridClock initialized: now 1786533608372477 us; error 31 us; skew 500 ppm
I20260812 11:20:08.374176 2063 webserver.cc:460] Webserver started at http://127.30.255.131:37743/ using document root <none> and password file <none>
I20260812 11:20:08.374753 2063 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:08.374822 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:08.375012 2063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:08.376919 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "7987bcb03ed965c080bf77d9f2f28ae9"
server_key_iv: "cdb48f972ce56299cbe931b32c5cd076"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.377441 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "7987bcb03ed965c080bf77d9f2f28ae9"
server_key_iv: "cdb48f972ce56299cbe931b32c5cd076"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.380887 2063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:08.383100 2087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.384232 2063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:08.384337 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90"
format_stamp: "Formatted at 2026-08-12 11:20:08 on dist-test-slave-gjrz"
server_key: "7987bcb03ed965c080bf77d9f2f28ae9"
server_key_iv: "cdb48f972ce56299cbe931b32c5cd076"
server_key_version: "encryptionkey@0"
I20260812 11:20:08.384425 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:08.395964 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:08.396512 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:08.396667 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:08.397164 2063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:08.397977 2063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:08.398034 2063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.398083 2063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:08.398108 2063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:08.406186 2063 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:39053
I20260812 11:20:08.406209 2315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:39053 every 8 connection(s)
I20260812 11:20:08.406939 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:20:08.408030 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2063
I20260812 11:20:08.408151 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:08.408416 31742 external_mini_cluster.cc:1428] Setting key 53ad969a14f34feaaa955df3d8d8a0c3
I20260812 11:20:08.414989 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.408613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45471 (local address 127.30.255.190:41185)
0812 11:20:08.408750 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.408755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.409844 (+ 1089us) server_negotiation.cc:407] Connection header received
0812 11:20:08.411110 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.411113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.411158 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.411223 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:08.412035 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.412539 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.413553 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.413753 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.413817 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.414645 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:08.414648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:08.414650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:08.414662 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:08.414678 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:08.414686 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:08.414739 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.414838 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.414885 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:08.415776 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.408986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41185 (local address 127.30.255.131:45471)
0812 11:20:08.409702 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.409750 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.410935 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.411400 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.411406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.411485 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:08.411930 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.411942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.412639 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.412641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.413468 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.413477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.413850 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.413853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.413980 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.414426 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:08.414427 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:08.414462 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:08.414521 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:08.414745 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:08.414750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:08.414753 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.414833 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.415028 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":146,"threads_started":1}
I20260812 11:20:08.416677 2316 heartbeater.cc:347] Connected to a master server at 127.30.255.190:41185
I20260812 11:20:08.416888 2316 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:08.417326 2316 heartbeater.cc:511] Master 127.30.255.190:41185 requested a full tablet report, sending...
I20260812 11:20:08.418251 1398 ts_manager.cc:194] Registered new tserver with Master: 7f7f0ee0d5b14c72b5afcbb96bc85c90 (127.30.255.131:39053)
I20260812 11:20:08.418766 1398 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.131:45471
I20260812 11:20:08.421803 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:08.426805 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.423736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.30.255.190:41185)
0812 11:20:08.423837 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.423841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.423996 (+ 155us) server_negotiation.cc:407] Connection header received
0812 11:20:08.424083 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.424085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.424117 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.424180 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:08.424378 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.424840 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.425650 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.425846 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.425916 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.426531 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:08.426534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:08.426536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:08.426546 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:08.426561 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:08.426568 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:08.426620 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.426676 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.426716 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:20:08.429189 1398 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:08.432139 1398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:08.445206 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.439975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:39685 (local address 127.0.0.1:44996)
0812 11:20:08.440697 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.440709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.440784 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.441401 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.441404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.441418 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:08.441560 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.441566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.442521 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.442524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.443781 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.443792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.444702 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.444705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.444808 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.444844 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.444949 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.445068 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":24,"spinlock_wait_cycles":1792,"thread_start_us":74,"threads_started":1}
I20260812 11:20:08.445214 2347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.440336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.30.255.130:39685)
0812 11:20:08.441022 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.441037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.441057 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:20:08.441123 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.441127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.441282 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.441419 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:08.441669 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.442407 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.443919 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.444585 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.444697 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.444869 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.444985 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.445074 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":129,"threads_started":1}
I20260812 11:20:08.445513 2344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.439753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39053 (local address 127.0.0.1:34542)
0812 11:20:08.440415 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.440442 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.440533 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.441329 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.441334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.441355 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:08.441511 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.441518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.443049 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.443052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.444205 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.444215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.445159 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.445163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.445250 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.445271 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.445320 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.445386 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":41,"thread_start_us":111,"threads_started":1}
I20260812 11:20:08.445726 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.439598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:39875 (local address 127.0.0.1:48730)
0812 11:20:08.440202 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.440249 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.440370 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.441437 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.441440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.441467 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:08.441605 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.441611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.443075 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.443078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.444122 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.444132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.445362 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.445365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.445451 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.445472 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.445550 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.445615 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":23,"spinlock_wait_cycles":768,"thread_start_us":60,"threads_started":1}
I20260812 11:20:08.446156 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.439720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.30.255.129:39875)
0812 11:20:08.440820 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.440839 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.440863 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:20:08.440951 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.440956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.441140 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.441281 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:08.441712 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.442925 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.444235 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.445024 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.445143 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.445305 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.445899 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.446001 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":144,"threads_started":1}
I20260812 11:20:08.447296 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.440314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.30.255.131:39053)
0812 11:20:08.440820 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.440832 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.440855 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:20:08.440921 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.440926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.441113 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.441649 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:08.441803 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.442955 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.444301 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.445024 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.446059 (+ 1035us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.446965 (+ 906us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.447024 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.447125 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":103,"threads_started":1}
I20260812 11:20:08.448225 1944 tablet_service.cc:1467] Processing CreateTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 (DEFAULT_TABLE table=test-table [id=3cc6ea11c51c481db339e377eb5bd424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:08.448869 1667 tablet_service.cc:1467] Processing CreateTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 (DEFAULT_TABLE table=test-table [id=3cc6ea11c51c481db339e377eb5bd424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:08.449357 1944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d7b62b1c944adc8b3bbfe83bb240b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:08.449293 2196 tablet_service.cc:1467] Processing CreateTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 (DEFAULT_TABLE table=test-table [id=3cc6ea11c51c481db339e377eb5bd424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:08.449894 1667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d7b62b1c944adc8b3bbfe83bb240b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:08.450122 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d7b62b1c944adc8b3bbfe83bb240b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:08.456027 2349 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Bootstrap starting.
I20260812 11:20:08.457121 2350 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Bootstrap starting.
I20260812 11:20:08.457486 2352 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Bootstrap starting.
I20260812 11:20:08.458235 2349 tablet_bootstrap.cc:654] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:08.458890 2349 log.cc:824] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:08.459187 2352 tablet_bootstrap.cc:654] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:08.459995 2352 log.cc:824] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:08.460923 2349 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: No bootstrap required, opened a new log
I20260812 11:20:08.461164 2349 ts_tablet_manager.cc:1397] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:20:08.461189 2350 tablet_bootstrap.cc:654] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:08.461841 2352 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: No bootstrap required, opened a new log
I20260812 11:20:08.461977 2350 log.cc:824] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:08.462040 2352 ts_tablet_manager.cc:1397] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:20:08.463930 2350 tablet_bootstrap.cc:492] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: No bootstrap required, opened a new log
I20260812 11:20:08.464121 2350 ts_tablet_manager.cc:1397] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:20:08.465230 2352 raft_consensus.cc:348] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.465255 2349 raft_consensus.cc:348] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.465435 2352 raft_consensus.cc:374] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:08.465494 2349 raft_consensus.cc:374] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:08.465508 2352 raft_consensus.cc:729] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b331ac799b4bd0a14c797b311329c1, State: Initialized, Role: FOLLOWER
I20260812 11:20:08.465590 2349 raft_consensus.cc:729] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7f0ee0d5b14c72b5afcbb96bc85c90, State: Initialized, Role: FOLLOWER
I20260812 11:20:08.465948 2352 consensus_queue.cc:260] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.466003 2349 consensus_queue.cc:260] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.466548 2059 heartbeater.cc:503] Master 127.30.255.190:41185 was elected leader, sending a full tablet report...
I20260812 11:20:08.466867 2352 ts_tablet_manager.cc:1428] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:20:08.466936 2349 ts_tablet_manager.cc:1428] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:08.467257 2316 heartbeater.cc:503] Master 127.30.255.190:41185 was elected leader, sending a full tablet report...
I20260812 11:20:08.468116 2350 raft_consensus.cc:348] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.468412 2350 raft_consensus.cc:374] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:08.468483 2350 raft_consensus.cc:729] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f60691e6eab4636a718c8ca64d1bef6, State: Initialized, Role: FOLLOWER
I20260812 11:20:08.468881 2350 consensus_queue.cc:260] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.469436 1805 heartbeater.cc:503] Master 127.30.255.190:41185 was elected leader, sending a full tablet report...
I20260812 11:20:08.469712 2350 ts_tablet_manager.cc:1428] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 11:20:08.488206 2060 tablet.cc:2369] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:20:08.562047 1807 tablet.cc:2369] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:20:08.657825 2317 tablet.cc:2369] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:08.678681 2362 raft_consensus.cc:482] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:08.678911 2362 raft_consensus.cc:504] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.680073 2362 leader_election.cc:288] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f7f0ee0d5b14c72b5afcbb96bc85c90 (127.30.255.131:39053), d6b331ac799b4bd0a14c797b311329c1 (127.30.255.130:39685)
I20260812 11:20:08.683821 2347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.680424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:38049 (local address 127.30.255.130:39685)
0812 11:20:08.680554 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.680559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.680730 (+ 171us) server_negotiation.cc:407] Connection header received
0812 11:20:08.680863 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.680866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.680910 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.680971 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:08.681252 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.681758 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.682882 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.683313 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.683421 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.683476 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.683629 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.683699 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:20:08.683844 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.680283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:39685 (local address 127.30.255.129:38049)
0812 11:20:08.680646 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.680658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.680764 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.680991 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.680994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.681010 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:08.681151 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.681157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.681861 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.681864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.682763 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.682773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.683422 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.683425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.683510 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.683533 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.683598 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.683667 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":20,"spinlock_wait_cycles":2944,"thread_start_us":103,"threads_started":1}
I20260812 11:20:08.683856 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.680390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:47449 (local address 127.30.255.131:39053)
0812 11:20:08.680533 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.680538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.680773 (+ 235us) server_negotiation.cc:407] Connection header received
0812 11:20:08.680906 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.680909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.680950 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.681016 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:08.681297 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.681776 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.682878 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.683345 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.683455 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.683504 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.683669 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.683737 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:20:08.683882 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.680284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:39053 (local address 127.30.255.129:47449)
0812 11:20:08.680683 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.680694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:08.680797 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:08.681028 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:08.681031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:08.681056 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:08.681197 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.681203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.681886 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.681890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.682763 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:08.682773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.683457 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.683461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:08.683562 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.683581 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:20:08.683645 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 11:20:08.683714 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":106,"spinlock_wait_cycles":768,"thread_start_us":81,"threads_started":1}
I20260812 11:20:08.684468 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d7b62b1c944adc8b3bbfe83bb240b7" candidate_uuid: "0f60691e6eab4636a718c8ca64d1bef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b331ac799b4bd0a14c797b311329c1" is_pre_election: true
I20260812 11:20:08.684578 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d7b62b1c944adc8b3bbfe83bb240b7" candidate_uuid: "0f60691e6eab4636a718c8ca64d1bef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" is_pre_election: true
I20260812 11:20:08.684751 1974 raft_consensus.cc:2454] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f60691e6eab4636a718c8ca64d1bef6 in term 0.
I20260812 11:20:08.684856 2226 raft_consensus.cc:2454] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f60691e6eab4636a718c8ca64d1bef6 in term 0.
I20260812 11:20:08.685266 1569 leader_election.cc:302] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f60691e6eab4636a718c8ca64d1bef6, d6b331ac799b4bd0a14c797b311329c1; no voters:
I20260812 11:20:08.685549 2362 raft_consensus.cc:2781] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:08.685653 2362 raft_consensus.cc:482] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:08.685700 2362 raft_consensus.cc:3037] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:08.686692 2362 raft_consensus.cc:504] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.686967 2362 leader_election.cc:288] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [CANDIDATE]: Term 1 election: Requested vote from peers 7f7f0ee0d5b14c72b5afcbb96bc85c90 (127.30.255.131:39053), d6b331ac799b4bd0a14c797b311329c1 (127.30.255.130:39685)
I20260812 11:20:08.687309 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d7b62b1c944adc8b3bbfe83bb240b7" candidate_uuid: "0f60691e6eab4636a718c8ca64d1bef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b331ac799b4bd0a14c797b311329c1"
I20260812 11:20:08.687428 1974 raft_consensus.cc:3037] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:08.688551 1974 raft_consensus.cc:2454] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f60691e6eab4636a718c8ca64d1bef6 in term 1.
I20260812 11:20:08.688844 1569 leader_election.cc:302] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f60691e6eab4636a718c8ca64d1bef6, d6b331ac799b4bd0a14c797b311329c1; no voters:
I20260812 11:20:08.688829 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d7b62b1c944adc8b3bbfe83bb240b7" candidate_uuid: "0f60691e6eab4636a718c8ca64d1bef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90"
I20260812 11:20:08.688946 2226 raft_consensus.cc:3037] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:08.689013 2362 raft_consensus.cc:2781] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:08.689229 2362 raft_consensus.cc:686] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [term 1 LEADER]: Becoming Leader. State: Replica: 0f60691e6eab4636a718c8ca64d1bef6, State: Running, Role: LEADER
I20260812 11:20:08.689538 2362 consensus_queue.cc:237] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } }
I20260812 11:20:08.690080 2226 raft_consensus.cc:2454] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f60691e6eab4636a718c8ca64d1bef6 in term 1.
I20260812 11:20:08.692735 1397 catalog_manager.cc:5495] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f60691e6eab4636a718c8ca64d1bef6 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "0f60691e6eab4636a718c8ca64d1bef6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f60691e6eab4636a718c8ca64d1bef6" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 39875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 } health_report { overall_health: UNKNOWN } } }
I20260812 11:20:08.732661 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.729530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.30.255.129:39875)
0812 11:20:08.729637 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:08.729642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:08.729658 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:20:08.729796 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:08.729799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:08.729855 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:08.729950 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:08.730146 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.730623 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.731446 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:08.731666 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:08.731753 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:08.731975 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:08.732376 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:08.732463 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:08.732509 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 11:20:08.732557 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:20:08.738132 2226 raft_consensus.cc:1261] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 1 FOLLOWER]: Refusing update from remote peer 0f60691e6eab4636a718c8ca64d1bef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:08.738152 1974 raft_consensus.cc:1261] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 1 FOLLOWER]: Refusing update from remote peer 0f60691e6eab4636a718c8ca64d1bef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:08.738783 2369 consensus_queue.cc:1035] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b331ac799b4bd0a14c797b311329c1" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:08.738998 2362 consensus_queue.cc:1035] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 0f60691e6eab4636a718c8ca64d1bef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f7f0ee0d5b14c72b5afcbb96bc85c90" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:08.746831 2384 mvcc.cc:205] Tried to move back new op lower bound from 7317641661384458240 to 7317641661194694656. Current Snapshot: MvccSnapshot[applied={T|T < 7317641661384458240}]
I20260812 11:20:08.754405 1398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:08.754539 1398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:08.756654 1398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2f72fa14a074d9a9f1e6ca85bbf5d3d: Sending DeleteTablet for 3 replicas of tablet 30d7b62b1c944adc8b3bbfe83bb240b7
I20260812 11:20:08.757369 2196 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:08 UTC) from {username='slave'} at 127.0.0.1:34542
I20260812 11:20:08.757400 1667 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:08 UTC) from {username='slave'} at 127.0.0.1:48730
I20260812 11:20:08.757499 1944 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d7b62b1c944adc8b3bbfe83bb240b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:08 UTC) from {username='slave'} at 127.0.0.1:44996
I20260812 11:20:08.757855 2393 tablet_replica.cc:330] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: stopping tablet replica
I20260812 11:20:08.757918 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1535
I20260812 11:20:08.757949 2397 tablet_replica.cc:330] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: stopping tablet replica
I20260812 11:20:08.761471 2393 raft_consensus.cc:2229] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:08.761476 2397 raft_consensus.cc:2229] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:08.761754 2393 raft_consensus.cc:2258] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:08.761754 2397 raft_consensus.cc:2258] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:08.765141 2393 ts_tablet_manager.cc:1905] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:08.765163 2397 ts_tablet_manager.cc:1905] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:08.767376 2393 ts_tablet_manager.cc:1918] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:08.767462 2393 log.cc:1196] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/30d7b62b1c944adc8b3bbfe83bb240b7
I20260812 11:20:08.767748 2393 ts_tablet_manager.cc:1939] T 30d7b62b1c944adc8b3bbfe83bb240b7 P 7f7f0ee0d5b14c72b5afcbb96bc85c90: Deleting consensus metadata
I20260812 11:20:08.767906 2397 ts_tablet_manager.cc:1918] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:08.767995 2397 log.cc:1196] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/30d7b62b1c944adc8b3bbfe83bb240b7
I20260812 11:20:08.768301 2397 ts_tablet_manager.cc:1939] T 30d7b62b1c944adc8b3bbfe83bb240b7 P d6b331ac799b4bd0a14c797b311329c1: Deleting consensus metadata
I20260812 11:20:08.768810 1366 catalog_manager.cc:4841] TS 7f7f0ee0d5b14c72b5afcbb96bc85c90 (127.30.255.131:39053): tablet 30d7b62b1c944adc8b3bbfe83bb240b7 (table test-table [id=3cc6ea11c51c481db339e377eb5bd424]) successfully deleted
I20260812 11:20:08.769313 1368 catalog_manager.cc:4841] TS d6b331ac799b4bd0a14c797b311329c1 (127.30.255.130:39685): tablet 30d7b62b1c944adc8b3bbfe83bb240b7 (table test-table [id=3cc6ea11c51c481db339e377eb5bd424]) successfully deleted
W20260812 11:20:08.773320 1371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:39875 (error 108)
I20260812 11:20:08.773466 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1809
I20260812 11:20:08.779397 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:08.779018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:39875 (local address 127.0.0.1:48756)
0812 11:20:08.779196 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:08.779310 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:39875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260812 11:20:08.779573 1371 catalog_manager.cc:4584] TS 0f60691e6eab4636a718c8ca64d1bef6 (127.30.255.129:39875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30d7b62b1c944adc8b3bbfe83bb240b7: Network error: Client connection negotiation failed: client connection to 127.30.255.129:39875: connect: Connection refused (error 111)
I20260812 11:20:08.789027 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2063
I20260812 11:20:08.801242 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 1336
2026-08-12T11:20:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1178 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:08 dist-test-slave-gjrz krb5kdc[2404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:08 dist-test-slave-gjrz krb5kdc[2404](info): set up 2 sockets
Aug 12 11:20:08 dist-test-slave-gjrz krb5kdc[2404](info): commencing operation
krb5kdc: starting...
W20260812 11:20:10.013239 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:10.127712 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37091
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:37091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:10.239512 2434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:10.239746 2434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:10.239789 2434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:10.243682 2434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:10.243728 2434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:10.243750 2434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:10.243779 2434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:10.243804 2434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:10.248813 2434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:37091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:10.249722 2434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:10.250634 2434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:10.258539 2452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.258570 2445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.258714 2434 server_base.cc:1034] running on GCE node
W20260812 11:20:10.258539 2442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.259629 2434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:10.260600 2434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:10.261759 2434 hybrid_clock.cc:648] HybridClock initialized: now 1786533610261738 us; error 31 us; skew 500 ppm
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:10.267753 2434 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:10.268544 2434 webserver.cc:460] Webserver started at http://127.30.255.190:37597/ using document root <none> and password file <none>
I20260812 11:20:10.269040 2434 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:10.269105 2434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:10.269311 2434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:10.271176 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "5930fd544ab44f5199974fd56c1ddba3"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "85f27768c882e0fbb11ae2597592fafc"
server_key_iv: "d8467a49879824943ad5ad289f7c0d52"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.271749 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "5930fd544ab44f5199974fd56c1ddba3"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "85f27768c882e0fbb11ae2597592fafc"
server_key_iv: "d8467a49879824943ad5ad289f7c0d52"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.275110 2434 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:10.277314 2463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.278425 2434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:10.278539 2434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "5930fd544ab44f5199974fd56c1ddba3"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "85f27768c882e0fbb11ae2597592fafc"
server_key_iv: "d8467a49879824943ad5ad289f7c0d52"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.278631 2434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:10.293489 2434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:10.296000 2434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:10.296168 2434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:10.302636 2434 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:37091
I20260812 11:20:10.302680 2568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:37091 every 8 connection(s)
I20260812 11:20:10.303550 2434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:10.306152 2570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:10.311434 2570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: Bootstrap starting.
I20260812 11:20:10.312978 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2434
I20260812 11:20:10.313117 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:10.313433 31742 external_mini_cluster.cc:1428] Setting key afd85d42e2a8cad19b30c8735fb8d0d6
I20260812 11:20:10.314025 2570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:10.314867 2570 log.cc:824] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:10.316844 2570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: No bootstrap required, opened a new log
I20260812 11:20:10.320708 2570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } }
I20260812 11:20:10.320982 2570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:10.321043 2570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5930fd544ab44f5199974fd56c1ddba3, State: Initialized, Role: FOLLOWER
I20260812 11:20:10.321529 2570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } }
I20260812 11:20:10.321672 2570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:10.321733 2570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:10.321827 2570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:10.322680 2570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } }
I20260812 11:20:10.323026 2570 leader_election.cc:302] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5930fd544ab44f5199974fd56c1ddba3; no voters:
I20260812 11:20:10.323369 2570 leader_election.cc:288] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:10.323436 2577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:10.323644 2577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [term 1 LEADER]: Becoming Leader. State: Replica: 5930fd544ab44f5199974fd56c1ddba3, State: Running, Role: LEADER
I20260812 11:20:10.323931 2577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } }
I20260812 11:20:10.324600 2570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:10.325574 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5930fd544ab44f5199974fd56c1ddba3. Latest consensus state: current_term: 1 leader_uuid: "5930fd544ab44f5199974fd56c1ddba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } } }
I20260812 11:20:10.325726 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:10.326128 2593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:10.326078 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5930fd544ab44f5199974fd56c1ddba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5930fd544ab44f5199974fd56c1ddba3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37091 } } }
I20260812 11:20:10.326313 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:10.329341 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.314491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.30.255.190:37091)
0812 11:20:10.314867 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:10.314882 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:10.314917 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:20:10.315690 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:10.315713 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:10.316013 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:10.316287 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:10.317205 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.318137 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.319382 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.319845 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.319957 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.322627 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:10.322632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:10.322645 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:10.322671 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:10.326263 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.326801 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.326807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.326814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.326959 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.327324 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.327327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.327329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.327811 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:10.328002 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:10.328310 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 11:20:10.328615 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
I20260812 11:20:10.331018 2593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:10.336108 2593 catalog_manager.cc:1347] Generated new cluster ID: 49fbbfc22fb94b59ad180add9b67c84a
I20260812 11:20:10.336218 2593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:10.354717 2593 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:10.355605 2593 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:10.365476 2593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: Generated new TSK 0
I20260812 11:20:10.366058 2593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:20:10.392475 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:10.500401 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:10.500614 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:10.500664 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:10.504555 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:10.504604 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:10.504673 2614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:10.509667 2614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:10.510545 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:10.511504 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:10.520226 2629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.520263 2623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.520262 2621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.520310 2614 server_base.cc:1034] running on GCE node
I20260812 11:20:10.520857 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:10.521404 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:10.522557 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786533610522540 us; error 25 us; skew 500 ppm
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:10.525163 2614 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:20:10.525987 2614 webserver.cc:460] Webserver started at http://127.30.255.129:37569/ using document root <none> and password file <none>
I20260812 11:20:10.526494 2614 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:10.526556 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:10.526717 2614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:10.528635 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "9ee87a0693f2486e92ff4d0d7ab08750"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "899c805c839dc1719fad1b1628300130"
server_key_iv: "1a68d88a28477424beb0b41f66be2ad7"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.529126 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "9ee87a0693f2486e92ff4d0d7ab08750"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "899c805c839dc1719fad1b1628300130"
server_key_iv: "1a68d88a28477424beb0b41f66be2ad7"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.532493 2614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:10.534665 2640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.535791 2614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:10.535888 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "9ee87a0693f2486e92ff4d0d7ab08750"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "899c805c839dc1719fad1b1628300130"
server_key_iv: "1a68d88a28477424beb0b41f66be2ad7"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.535981 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:10.548403 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:10.551122 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:10.551299 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:10.551805 2614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:10.552536 2614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:10.552583 2614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.552623 2614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:10.552650 2614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.561033 2614 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:42231
I20260812 11:20:10.561110 2868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:42231 every 8 connection(s)
I20260812 11:20:10.561869 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:20:10.568063 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2614
I20260812 11:20:10.568181 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:10.568444 31742 external_mini_cluster.cc:1428] Setting key a3b6aa76a9b7eb5bb587313c021a2b1a
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:10.574892 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.563529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:55587 (local address 127.30.255.190:37091)
0812 11:20:10.563660 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:10.563664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:10.564423 (+ 759us) server_negotiation.cc:407] Connection header received
0812 11:20:10.565690 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:10.565694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:10.565751 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:10.565827 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:10.566909 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.567495 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.568699 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.568970 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.569070 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.571579 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:10.571582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:10.571584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:10.571608 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:10.573000 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.573459 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.573463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.573464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.573559 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.573851 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.573854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.573856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.574060 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:10.574157 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:10.574586 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 11:20:10.574731 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:10.575537 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.563851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37091 (local address 127.30.255.129:55587)
0812 11:20:10.564287 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:10.564327 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:10.565484 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:10.565994 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:10.566001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:10.566318 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:10.566763 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.566776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.567616 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.567620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.568571 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.568582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.569124 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.569128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.569312 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.569823 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:10.569838 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:10.571422 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:10.573110 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.573115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.573125 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.573360 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.573652 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.573654 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.573655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.573770 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.574145 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:10.574149 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:10.574351 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 11:20:10.574494 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 11:20:10.574711 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
I20260812 11:20:10.576596 2871 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37091
I20260812 11:20:10.576793 2871 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:10.577207 2871 heartbeater.cc:511] Master 127.30.255.190:37091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:20:10.578583 2494 ts_manager.cc:194] Registered new tserver with Master: 9ee87a0693f2486e92ff4d0d7ab08750 (127.30.255.129:42231)
I20260812 11:20:10.579790 2494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:55587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:20:10.594426 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:10.700766 2881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:10.700987 2881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:10.701030 2881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:10.704890 2881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:10.704937 2881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:10.705004 2881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:10.710325 2881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:10.711230 2881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:10.712177 2881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:10.721112 2893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.721100 2889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.721102 2888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.721369 2881 server_base.cc:1034] running on GCE node
I20260812 11:20:10.721737 2881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:10.722316 2881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:10.723470 2881 hybrid_clock.cc:648] HybridClock initialized: now 1786533610723457 us; error 27 us; skew 500 ppm
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:10.726217 2881 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:20:10.727029 2881 webserver.cc:460] Webserver started at http://127.30.255.130:44977/ using document root <none> and password file <none>
I20260812 11:20:10.727586 2881 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:10.727654 2881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:10.727854 2881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:10.729820 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "f1616aa1cb034468ab18287571559f7a"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "a6a05c3a31f26361649047cf34be8609"
server_key_iv: "ed6fcdd0e2d2989e7fa780cc75d698ae"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.730350 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "f1616aa1cb034468ab18287571559f7a"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "a6a05c3a31f26361649047cf34be8609"
server_key_iv: "ed6fcdd0e2d2989e7fa780cc75d698ae"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.733862 2881 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 11:20:10.736169 2906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.737285 2881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:10.737388 2881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "f1616aa1cb034468ab18287571559f7a"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "a6a05c3a31f26361649047cf34be8609"
server_key_iv: "ed6fcdd0e2d2989e7fa780cc75d698ae"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.737478 2881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:10.751152 2881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:10.753957 2881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:10.754125 2881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:10.754638 2881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:10.755424 2881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:10.755471 2881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.755520 2881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:10.755548 2881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.763831 2881 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:34379
I20260812 11:20:10.763967 3129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:34379 every 8 connection(s)
I20260812 11:20:10.764642 2881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:10.769506 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 2881
I20260812 11:20:10.769642 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:10.769939 31742 external_mini_cluster.cc:1428] Setting key 8c8a76101bd8494b4eba6de51e94ac23
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:10.777956 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.766263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:37273 (local address 127.30.255.190:37091)
0812 11:20:10.766385 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:10.766389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:10.767151 (+ 762us) server_negotiation.cc:407] Connection header received
0812 11:20:10.768416 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:10.768419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:10.768466 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:10.768527 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:10.769673 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.770248 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.771635 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.771854 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.771931 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.774601 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:10.774603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:10.774605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:10.774630 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:10.775845 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.776447 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.776450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.776451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.776525 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.776866 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.776869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.776871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.777080 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:10.777156 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:10.777733 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 11:20:10.777827 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:10.778965 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.766600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37091 (local address 127.30.255.130:37273)
0812 11:20:10.767014 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:10.767054 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:10.768223 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:10.768665 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:10.768672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:10.768975 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:10.769510 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.769531 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.770373 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.770376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.771524 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.771536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.771995 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.771999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.772183 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.772685 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:10.772705 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:10.774440 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:10.775960 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.775967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.775982 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.776348 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.776608 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.776611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.776613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.776782 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.777189 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:10.777196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:10.777490 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 11:20:10.777702 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 11:20:10.778020 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":108,"threads_started":1}
I20260812 11:20:10.780082 3130 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37091
I20260812 11:20:10.780318 3130 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:20:10.780849 3130 heartbeater.cc:511] Master 127.30.255.190:37091 requested a full tablet report, sending...
I20260812 11:20:10.781831 2494 ts_manager.cc:194] Registered new tserver with Master: f1616aa1cb034468ab18287571559f7a (127.30.255.130:34379)
I20260812 11:20:10.782416 2494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:37273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:20:10.796034 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:10.900733 3140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:10.900941 3140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:10.900979 3140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:10.904909 3140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:10.904959 3140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:10.905012 3140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:10.909945 3140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:10.910771 3140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:10.911657 3140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:10.920379 3153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:10.920372 3151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.920583 3140 server_base.cc:1034] running on GCE node
W20260812 11:20:10.920576 3159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:10.921061 3140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:10.921597 3140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:10.922766 3140 hybrid_clock.cc:648] HybridClock initialized: now 1786533610922749 us; error 26 us; skew 500 ppm
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:10.925396 3140 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:20:10.926277 3140 webserver.cc:460] Webserver started at http://127.30.255.131:41117/ using document root <none> and password file <none>
I20260812 11:20:10.926810 3140 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:10.926870 3140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:10.927100 3140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:10.929015 3140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "d21d9875802d40319d84abefc7f93633"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "f59f1b85f53633a8709f0078517e0987"
server_key_iv: "f1c9ed9976b7c93718508c861dad94de"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.929520 3140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "d21d9875802d40319d84abefc7f93633"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "f59f1b85f53633a8709f0078517e0987"
server_key_iv: "f1c9ed9976b7c93718508c861dad94de"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.932940 3140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:10.935022 3169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.936148 3140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:10.936265 3140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "d21d9875802d40319d84abefc7f93633"
format_stamp: "Formatted at 2026-08-12 11:20:10 on dist-test-slave-gjrz"
server_key: "f59f1b85f53633a8709f0078517e0987"
server_key_iv: "f1c9ed9976b7c93718508c861dad94de"
server_key_version: "encryptionkey@0"
I20260812 11:20:10.936353 3140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:10.948483 3140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:10.950891 3140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:10.951040 3140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:10.951604 3140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:10.952332 3140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:10.952379 3140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.952428 3140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:10.952457 3140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:10.960654 3140 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:37219
I20260812 11:20:10.960683 3374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:37219 every 8 connection(s)
I20260812 11:20:10.961441 3140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
Aug 12 11:20:10 dist-test-slave-gjrz krb5kdc[2404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533610, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:10.971248 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3140
I20260812 11:20:10.971408 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:10.971724 31742 external_mini_cluster.cc:1428] Setting key dfb531afdf1c19825ab52a527b5423ad
I20260812 11:20:10.973660 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.962979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:46459 (local address 127.30.255.190:37091)
0812 11:20:10.963102 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:10.963106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:10.963819 (+ 713us) server_negotiation.cc:407] Connection header received
0812 11:20:10.965066 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:10.965070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:10.965129 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:10.965207 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:10.966260 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.966796 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.967957 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.968191 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.968284 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.970367 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:10.970371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:10.970374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:10.970398 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:10.971684 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.972172 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.972176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.972178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.972267 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.972590 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.972594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.972595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.972795 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:10.972887 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:10.973395 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 11:20:10.973513 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:20:10.974293 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.963244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37091 (local address 127.30.255.131:46459)
0812 11:20:10.963677 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:10.963716 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:10.964859 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:10.965373 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:10.965380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:10.965683 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:10.966106 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.966129 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.966916 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.966919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.967830 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:10.967841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.968328 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.968333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:10.968485 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.968906 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:10.968921 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:10.970209 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:10.971795 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.971802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.971816 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.972068 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.972353 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:10.972356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:10.972358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:10.972503 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:10.972895 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:10.972901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:10.973172 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 11:20:10.973318 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 11:20:10.973537 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":104,"threads_started":1}
I20260812 11:20:10.975236 3375 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37091
I20260812 11:20:10.975435 3375 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:10.975849 3375 heartbeater.cc:511] Master 127.30.255.190:37091 requested a full tablet report, sending...
I20260812 11:20:10.976739 2494 ts_manager.cc:194] Registered new tserver with Master: d21d9875802d40319d84abefc7f93633 (127.30.255.131:37219)
I20260812 11:20:10.977258 2494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:46459
I20260812 11:20:10.985244 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:10.993160 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:10.986998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.30.255.190:37091)
0812 11:20:10.987103 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:10.987106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:10.987284 (+ 178us) server_negotiation.cc:407] Connection header received
0812 11:20:10.987391 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:10.987393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:10.987428 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:10.987485 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:10.988031 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.988491 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.989194 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:10.989382 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:10.989443 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:10.990719 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:10.990722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:10.990724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:10.990745 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:10.991841 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.992254 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.992256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.992257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.992321 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:10.992540 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:10.992542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:10.992543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:10.992710 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:10.992812 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:10.992985 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 11:20:10.993053 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:10.995616 2494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:10.998355 2494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:11.011494 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.005925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.30.255.130:34379)
0812 11:20:11.006577 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.006590 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.006615 (+ 25us) server_negotiation.cc:407] Connection header received
0812 11:20:11.006693 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.006698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.006889 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.007043 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:11.007368 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.008469 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.009907 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.010564 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.010662 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.010785 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.011224 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.011349 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":142,"threads_started":1}
I20260812 11:20:11.011646 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.006720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.30.255.131:37219)
0812 11:20:11.007424 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.007435 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.007454 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:20:11.007514 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.007519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.007695 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.007827 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:11.008096 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.008943 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.010518 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.011060 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.011167 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.011303 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.011430 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.011513 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":128,"threads_started":1}
I20260812 11:20:11.011731 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.006373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:37219 (local address 127.0.0.1:44860)
0812 11:20:11.007096 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.007109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.007190 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.007901 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.007904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.007921 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:11.007989 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.007993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.009080 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.009083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.010404 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.010412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.011178 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.011182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.011302 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.011326 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.011392 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.011514 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":148,"thread_start_us":84,"threads_started":1}
I20260812 11:20:11.012073 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.005742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:34379 (local address 127.0.0.1:46764)
0812 11:20:11.006055 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.006103 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.006220 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.007074 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.007080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.007102 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:11.007227 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.007235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.008604 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.008607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.009767 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.009776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.010888 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.010891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.011002 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.011038 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.011915 (+ 877us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.011974 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":57,"threads_started":1}
I20260812 11:20:11.012072 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.006116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:42231 (local address 127.0.0.1:53690)
0812 11:20:11.006373 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.006386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.006469 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.007744 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.007747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.007764 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:11.007854 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.007860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.009040 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.009043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.010348 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.010359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.011685 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.011689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.011779 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.011802 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.011873 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.011937 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":28,"thread_start_us":110,"threads_started":1}
I20260812 11:20:11.012328 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.006728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.30.255.129:42231)
0812 11:20:11.007393 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.007405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.007424 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:20:11.007483 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.007487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.007615 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.007741 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:11.007978 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.008921 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.010491 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.011360 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.011472 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.011601 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.012098 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.012198 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":130,"threads_started":1}
I20260812 11:20:11.014421 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa (DEFAULT_TABLE table=test-table [id=80ea3b870e6643f28c3629928438595b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:11.014914 2745 tablet_service.cc:1467] Processing CreateTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa (DEFAULT_TABLE table=test-table [id=80ea3b870e6643f28c3629928438595b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:11.014914 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa (DEFAULT_TABLE table=test-table [id=80ea3b870e6643f28c3629928438595b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:11.015430 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:11.015944 2745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:11.015944 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:11.021176 3403 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Bootstrap starting.
I20260812 11:20:11.022804 3405 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Bootstrap starting.
I20260812 11:20:11.023187 3404 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Bootstrap starting.
I20260812 11:20:11.023552 3403 tablet_bootstrap.cc:654] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:11.024837 3403 log.cc:824] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:11.025398 3404 tablet_bootstrap.cc:654] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:11.026623 3404 log.cc:824] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:11.026805 3403 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: No bootstrap required, opened a new log
I20260812 11:20:11.026988 3403 ts_tablet_manager.cc:1397] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 11:20:11.027297 3405 tablet_bootstrap.cc:654] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:11.028266 3405 log.cc:824] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:11.028558 3404 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: No bootstrap required, opened a new log
I20260812 11:20:11.028756 3404 ts_tablet_manager.cc:1397] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:11.029970 3405 tablet_bootstrap.cc:492] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: No bootstrap required, opened a new log
I20260812 11:20:11.030109 3405 ts_tablet_manager.cc:1397] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:20:11.030691 3403 raft_consensus.cc:348] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.030928 3403 raft_consensus.cc:374] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:11.030982 3403 raft_consensus.cc:729] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d21d9875802d40319d84abefc7f93633, State: Initialized, Role: FOLLOWER
I20260812 11:20:11.031450 3403 consensus_queue.cc:260] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.032097 3375 heartbeater.cc:503] Master 127.30.255.190:37091 was elected leader, sending a full tablet report...
I20260812 11:20:11.032450 3403 ts_tablet_manager.cc:1428] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:20:11.032384 3404 raft_consensus.cc:348] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.032610 3404 raft_consensus.cc:374] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:11.032671 3404 raft_consensus.cc:729] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee87a0693f2486e92ff4d0d7ab08750, State: Initialized, Role: FOLLOWER
I20260812 11:20:11.033318 3404 consensus_queue.cc:260] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.033285 3405 raft_consensus.cc:348] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.033571 3405 raft_consensus.cc:374] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:11.033660 3405 raft_consensus.cc:729] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1616aa1cb034468ab18287571559f7a, State: Initialized, Role: FOLLOWER
I20260812 11:20:11.033928 2871 heartbeater.cc:503] Master 127.30.255.190:37091 was elected leader, sending a full tablet report...
I20260812 11:20:11.034222 3404 ts_tablet_manager.cc:1428] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 11:20:11.035168 3405 consensus_queue.cc:260] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.035946 3130 heartbeater.cc:503] Master 127.30.255.190:37091 was elected leader, sending a full tablet report...
I20260812 11:20:11.036136 3405 ts_tablet_manager.cc:1428] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:11.040380 3418 raft_consensus.cc:482] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:11.040524 3418 raft_consensus.cc:504] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.041471 3418 leader_election.cc:288] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1616aa1cb034468ab18287571559f7a (127.30.255.130:34379), d21d9875802d40319d84abefc7f93633 (127.30.255.131:37219)
I20260812 11:20:11.044982 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.041679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:34379 (local address 127.30.255.129:34663)
0812 11:20:11.041793 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.041808 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.041906 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.042280 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.042283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.042301 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:11.042432 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.042451 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.043129 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.043133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.043949 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.043959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.044558 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.044673 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.044700 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.044776 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.044853 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"spinlock_wait_cycles":21120}
I20260812 11:20:11.045001 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.041724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:34663 (local address 127.30.255.130:34379)
0812 11:20:11.041832 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.041837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.041850 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:11.042041 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.042045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.042090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.042156 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:11.042545 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.043032 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044065 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.044447 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044562 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.044611 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.044821 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.044891 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:11.045385 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.042037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:60943 (local address 127.30.255.131:37219)
0812 11:20:11.042140 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.042144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.042316 (+ 172us) server_negotiation.cc:407] Connection header received
0812 11:20:11.042429 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.042431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.042467 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.042537 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:11.042803 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.043255 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044283 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.044773 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044867 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.044909 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.045209 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.045285 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 11:20:11.045645 3051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f1f8a24abf4d56b7bf5eefc3a880fa" candidate_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1616aa1cb034468ab18287571559f7a" is_pre_election: true
I20260812 11:20:11.045742 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.041931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:37219 (local address 127.30.255.129:60943)
0812 11:20:11.042227 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.042240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.042350 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.042565 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.042569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.042586 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:11.042706 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.042711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.043383 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.043387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.044168 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.044178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.044965 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.044968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.045061 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.045086 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.045518 (+ 432us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.045596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"spinlock_wait_cycles":384,"thread_start_us":67,"threads_started":1}
I20260812 11:20:11.045893 3051 raft_consensus.cc:2454] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ee87a0693f2486e92ff4d0d7ab08750 in term 0.
I20260812 11:20:11.046309 2644 leader_election.cc:302] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ee87a0693f2486e92ff4d0d7ab08750, f1616aa1cb034468ab18287571559f7a; no voters:
I20260812 11:20:11.046537 3418 raft_consensus.cc:2781] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:11.046506 3299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f1f8a24abf4d56b7bf5eefc3a880fa" candidate_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d21d9875802d40319d84abefc7f93633" is_pre_election: true
I20260812 11:20:11.046623 3418 raft_consensus.cc:482] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:11.046675 3418 raft_consensus.cc:3037] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:11.046758 3299 raft_consensus.cc:2454] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ee87a0693f2486e92ff4d0d7ab08750 in term 0.
I20260812 11:20:11.047677 3418 raft_consensus.cc:504] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.047972 3418 leader_election.cc:288] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [CANDIDATE]: Term 1 election: Requested vote from peers f1616aa1cb034468ab18287571559f7a (127.30.255.130:34379), d21d9875802d40319d84abefc7f93633 (127.30.255.131:37219)
I20260812 11:20:11.048261 3051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f1f8a24abf4d56b7bf5eefc3a880fa" candidate_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1616aa1cb034468ab18287571559f7a"
I20260812 11:20:11.048377 3299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f1f8a24abf4d56b7bf5eefc3a880fa" candidate_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d21d9875802d40319d84abefc7f93633"
I20260812 11:20:11.048403 3051 raft_consensus.cc:3037] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:11.048483 3299 raft_consensus.cc:3037] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:11.049582 3299 raft_consensus.cc:2454] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ee87a0693f2486e92ff4d0d7ab08750 in term 1.
I20260812 11:20:11.049582 3051 raft_consensus.cc:2454] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ee87a0693f2486e92ff4d0d7ab08750 in term 1.
I20260812 11:20:11.049871 2644 leader_election.cc:302] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ee87a0693f2486e92ff4d0d7ab08750, d21d9875802d40319d84abefc7f93633; no voters:
I20260812 11:20:11.050027 3418 raft_consensus.cc:2781] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:11.050215 3418 raft_consensus.cc:686] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [term 1 LEADER]: Becoming Leader. State: Replica: 9ee87a0693f2486e92ff4d0d7ab08750, State: Running, Role: LEADER
I20260812 11:20:11.050473 3418 consensus_queue.cc:237] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } }
I20260812 11:20:11.053097 2493 catalog_manager.cc:5495] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ee87a0693f2486e92ff4d0d7ab08750 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ee87a0693f2486e92ff4d0d7ab08750" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 42231 } health_report { overall_health: HEALTHY } } }
I20260812 11:20:11.062533 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.059310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.30.255.129:42231)
0812 11:20:11.059405 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.059409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.059423 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:20:11.059580 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.059583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.059628 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.059707 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:11.059947 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.060410 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.061215 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.061416 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.061496 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.061805 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:11.062166 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:11.062288 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.062393 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.062426 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 11:20:11.062855 2872 tablet.cc:2369] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:11.067858 3299 raft_consensus.cc:1261] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 1 FOLLOWER]: Refusing update from remote peer 9ee87a0693f2486e92ff4d0d7ab08750: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:11.067895 3051 raft_consensus.cc:1261] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 1 FOLLOWER]: Refusing update from remote peer 9ee87a0693f2486e92ff4d0d7ab08750: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:11.068526 3418 consensus_queue.cc:1035] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1616aa1cb034468ab18287571559f7a" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:11.068710 3423 consensus_queue.cc:1035] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d21d9875802d40319d84abefc7f93633" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 37219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:11.083323 2494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:11.083463 2494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:11.085614 2494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5930fd544ab44f5199974fd56c1ddba3: Sending DeleteTablet for 3 replicas of tablet 19f1f8a24abf4d56b7bf5eefc3a880fa
I20260812 11:20:11.086440 2745 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:11 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:53690
I20260812 11:20:11.086476 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:11 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:46764
I20260812 11:20:11.086530 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:11 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:44860
I20260812 11:20:11.087162 3458 tablet_replica.cc:330] T 19f1f8a24abf4d56b7bf5eefc3a880fa P 9ee87a0693f2486e92ff4d0d7ab08750: stopping tablet replica
I20260812 11:20:11.087402 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2614
I20260812 11:20:11.098680 3460 tablet_replica.cc:330] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: stopping tablet replica
I20260812 11:20:11.098968 3460 raft_consensus.cc:2229] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:11.099174 3462 tablet_replica.cc:330] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: stopping tablet replica
I20260812 11:20:11.099244 3460 raft_consensus.cc:2258] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:11.099478 3462 raft_consensus.cc:2229] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:11.099776 3462 raft_consensus.cc:2258] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:11.101818 3460 ts_tablet_manager.cc:1905] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:11.103391 3462 ts_tablet_manager.cc:1905] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:11.104548 3460 ts_tablet_manager.cc:1918] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:11.104715 3460 log.cc:1196] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/19f1f8a24abf4d56b7bf5eefc3a880fa
I20260812 11:20:11.105108 3460 ts_tablet_manager.cc:1939] T 19f1f8a24abf4d56b7bf5eefc3a880fa P f1616aa1cb034468ab18287571559f7a: Deleting consensus metadata
I20260812 11:20:11.106026 3462 ts_tablet_manager.cc:1918] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:11.106134 3462 log.cc:1196] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/19f1f8a24abf4d56b7bf5eefc3a880fa
I20260812 11:20:11.106386 2464 catalog_manager.cc:4841] TS f1616aa1cb034468ab18287571559f7a (127.30.255.130:34379): tablet 19f1f8a24abf4d56b7bf5eefc3a880fa (table test-table [id=80ea3b870e6643f28c3629928438595b]) successfully deleted
I20260812 11:20:11.106488 3462 ts_tablet_manager.cc:1939] T 19f1f8a24abf4d56b7bf5eefc3a880fa P d21d9875802d40319d84abefc7f93633: Deleting consensus metadata
I20260812 11:20:11.107447 2464 catalog_manager.cc:4841] TS d21d9875802d40319d84abefc7f93633 (127.30.255.131:37219): tablet 19f1f8a24abf4d56b7bf5eefc3a880fa (table test-table [id=80ea3b870e6643f28c3629928438595b]) successfully deleted
W20260812 11:20:11.109364 2475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:42231 (error 108)
I20260812 11:20:11.109459 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2881
I20260812 11:20:11.120426 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.120126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:42231 (local address 127.0.0.1:53710)
0812 11:20:11.120287 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.120347 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:42231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260812 11:20:11.120617 2475 catalog_manager.cc:4584] TS 9ee87a0693f2486e92ff4d0d7ab08750 (127.30.255.129:42231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19f1f8a24abf4d56b7bf5eefc3a880fa: Network error: Client connection negotiation failed: client connection to 127.30.255.129:42231: connect: Connection refused (error 111)
I20260812 11:20:11.129456 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3140
I20260812 11:20:11.154891 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 2434
2026-08-12T11:20:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2355 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T11:20:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:11Z Disabled control of system clock
I20260812 11:20:11.189451 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42239
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:36115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:42239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:11.295639 3472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:11.295855 3472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:11.295887 3472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:11.299703 3472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:11.299748 3472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:11.299767 3472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:11.299782 3472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:11.299795 3472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:11.304314 3472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:42239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:42239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:11.305233 3472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:11.306468 3472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:11.314507 3484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.314517 3477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.314512 3479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.314714 3472 server_base.cc:1034] running on GCE node
I20260812 11:20:11.315348 3472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:11.316339 3472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:11.317497 3472 hybrid_clock.cc:648] HybridClock initialized: now 1786533611317481 us; error 26 us; skew 500 ppm
I20260812 11:20:11.319190 3472 webserver.cc:460] Webserver started at http://127.30.255.190:33037/ using document root <none> and password file <none>
I20260812 11:20:11.319753 3472 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:11.319818 3472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:11.320001 3472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:11.321934 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "648c250bbf454b2b818b957f7734578d"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "df832f05e9d282b9fbb90c0a16f8e7cd"
server_key_iv: "7c9161a7e7887ce035ac8d287913ccfe"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.322458 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "648c250bbf454b2b818b957f7734578d"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "df832f05e9d282b9fbb90c0a16f8e7cd"
server_key_iv: "7c9161a7e7887ce035ac8d287913ccfe"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.325946 3472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:20:11.328215 3496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.329329 3472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:11.329450 3472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "648c250bbf454b2b818b957f7734578d"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "df832f05e9d282b9fbb90c0a16f8e7cd"
server_key_iv: "7c9161a7e7887ce035ac8d287913ccfe"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.329540 3472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:11.348243 3472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:11.348827 3472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:11.348986 3472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:11.355594 3472 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:42239
I20260812 11:20:11.355605 3609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:42239 every 8 connection(s)
I20260812 11:20:11.356448 3472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:11.359488 3612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:11.364451 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3472
I20260812 11:20:11.364570 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:11.364820 31742 external_mini_cluster.cc:1428] Setting key f5a9052fc3f8a893d19326203cd2cde7
I20260812 11:20:11.364933 3612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: Bootstrap starting.
I20260812 11:20:11.367239 3612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:11.368002 3612 log.cc:824] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:11.369889 3612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: No bootstrap required, opened a new log
I20260812 11:20:11.373675 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.366093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.30.255.190:42239)
0812 11:20:11.366695 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.366709 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.366743 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:20:11.367394 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.367413 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.367807 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.368066 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:11.368445 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.369504 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.370681 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.371135 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.371301 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.372103 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:11.372108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:11.372119 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:11.372464 (+ 345us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:11.372517 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:11.372536 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:11.372611 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.372757 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.373001 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
I20260812 11:20:11.373710 3612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } }
I20260812 11:20:11.373958 3612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:11.374029 3612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648c250bbf454b2b818b957f7734578d, State: Initialized, Role: FOLLOWER
I20260812 11:20:11.374420 3612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } }
I20260812 11:20:11.374548 3612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:11.374612 3612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:11.374714 3612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:11.375707 3612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } }
I20260812 11:20:11.376060 3612 leader_election.cc:302] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 648c250bbf454b2b818b957f7734578d; no voters:
I20260812 11:20:11.376348 3612 leader_election.cc:288] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:11.376417 3621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:11.376628 3621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [term 1 LEADER]: Becoming Leader. State: Replica: 648c250bbf454b2b818b957f7734578d, State: Running, Role: LEADER
I20260812 11:20:11.376895 3621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } }
I20260812 11:20:11.377424 3612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:11.378295 3623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "648c250bbf454b2b818b957f7734578d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } } }
I20260812 11:20:11.378437 3623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:11.378803 3638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:11.378780 3624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648c250bbf454b2b818b957f7734578d. Latest consensus state: current_term: 1 leader_uuid: "648c250bbf454b2b818b957f7734578d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c250bbf454b2b818b957f7734578d" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 42239 } } }
I20260812 11:20:11.378981 3624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:11.381815 3638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:11.386929 3638 catalog_manager.cc:1347] Generated new cluster ID: bfa2f32ed6694529ad193ddb4be80586
I20260812 11:20:11.386993 3638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:11.400321 3638 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:11.401366 3638 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:11.409629 3638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: Generated new TSK 0
I20260812 11:20:11.410295 3638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:11.413985 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--builtin_ntp_servers=127.30.255.148:36115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:11.569442 3672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:11.569672 3672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:11.569707 3672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:11.573607 3672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:11.573658 3672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:11.573709 3672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:11.578266 3672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:11.579208 3672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:11.580492 3672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:11.589579 3682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.589586 3690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.589639 3687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.590016 3672 server_base.cc:1034] running on GCE node
I20260812 11:20:11.590385 3672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:11.590948 3672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:11.592128 3672 hybrid_clock.cc:648] HybridClock initialized: now 1786533611592105 us; error 31 us; skew 500 ppm
I20260812 11:20:11.593812 3672 webserver.cc:460] Webserver started at http://127.30.255.129:36649/ using document root <none> and password file <none>
I20260812 11:20:11.594342 3672 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:11.594408 3672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:11.594594 3672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:11.596565 3672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "d6e8d73209c74d5b89ef3e08a9c04337"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8302b33923fdc65b61f2433e48259760"
server_key_iv: "754b46e6ede1a8fe13486b1e23616d8a"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.597091 3672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "d6e8d73209c74d5b89ef3e08a9c04337"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8302b33923fdc65b61f2433e48259760"
server_key_iv: "754b46e6ede1a8fe13486b1e23616d8a"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.600517 3672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:11.602751 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.603878 3672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:11.603997 3672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "d6e8d73209c74d5b89ef3e08a9c04337"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8302b33923fdc65b61f2433e48259760"
server_key_iv: "754b46e6ede1a8fe13486b1e23616d8a"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.604086 3672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:11.630906 3672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:11.631541 3672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:11.631699 3672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:11.632256 3672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:11.633168 3672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:11.633216 3672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.633265 3672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:11.633292 3672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.641137 3672 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:35937
I20260812 11:20:11.641179 3928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:35937 every 8 connection(s)
I20260812 11:20:11.641947 3672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:20:11.643607 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3672
I20260812 11:20:11.643738 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:11.644029 31742 external_mini_cluster.cc:1428] Setting key a928991309d7ec714bd86914620fbd4a
I20260812 11:20:11.645895 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--builtin_ntp_servers=127.30.255.148:36115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:11.651607 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.643612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:48705 (local address 127.30.255.190:42239)
0812 11:20:11.643734 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.643737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.645055 (+ 1318us) server_negotiation.cc:407] Connection header received
0812 11:20:11.646438 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.646441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.646499 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.646573 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:11.647546 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.648076 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.649620 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.649854 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.649943 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.651162 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:11.651165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:11.651166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:11.651179 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:11.651198 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:11.651205 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:11.651285 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.651446 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.651496 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:20:11.652722 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.644165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:42239 (local address 127.30.255.129:48705)
0812 11:20:11.644896 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.644947 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.646224 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.646749 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.646756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.646839 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:11.647359 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.647382 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.648197 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.648201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.649486 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.649499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.650004 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.650008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.650230 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.650866 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:11.650868 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:11.650924 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:11.651007 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:11.651302 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:11.651311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:11.651318 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.651452 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.651741 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":139,"threads_started":1}
I20260812 11:20:11.653911 3929 heartbeater.cc:347] Connected to a master server at 127.30.255.190:42239
I20260812 11:20:11.654235 3929 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:11.654744 3929 heartbeater.cc:511] Master 127.30.255.190:42239 requested a full tablet report, sending...
I20260812 11:20:11.656280 3541 ts_manager.cc:194] Registered new tserver with Master: d6e8d73209c74d5b89ef3e08a9c04337 (127.30.255.129:35937)
I20260812 11:20:11.657392 3541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.129:48705
W20260812 11:20:11.755288 3934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:11.755522 3934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:11.755573 3934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:11.759414 3934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:11.759459 3934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:11.759531 3934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:11.764107 3934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:11.764983 3934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:11.766196 3934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:11.775199 3945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.775254 3934 server_base.cc:1034] running on GCE node
W20260812 11:20:11.775238 3952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.775301 3944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.775909 3934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:11.776490 3934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:11.777649 3934 hybrid_clock.cc:648] HybridClock initialized: now 1786533611777633 us; error 30 us; skew 500 ppm
I20260812 11:20:11.779420 3934 webserver.cc:460] Webserver started at http://127.30.255.130:39623/ using document root <none> and password file <none>
I20260812 11:20:11.779953 3934 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:11.780021 3934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:11.780202 3934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:11.782166 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "a7c923c15e904df99d163e299d263fdb"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8cc26afb00ddbe359589b6dd4fd86216"
server_key_iv: "99e691d45ffe455c121d324c23a0445f"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.782709 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "a7c923c15e904df99d163e299d263fdb"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8cc26afb00ddbe359589b6dd4fd86216"
server_key_iv: "99e691d45ffe455c121d324c23a0445f"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.786163 3934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 11:20:11.788414 3960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.789497 3934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:11.789611 3934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "a7c923c15e904df99d163e299d263fdb"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "8cc26afb00ddbe359589b6dd4fd86216"
server_key_iv: "99e691d45ffe455c121d324c23a0445f"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.789695 3934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:11.814862 3934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:11.815441 3934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:11.815600 3934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:11.816115 3934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:11.816998 3934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:11.817045 3934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.817093 3934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:11.817121 3934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.825285 3934 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:40009
I20260812 11:20:11.825318 4180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:40009 every 8 connection(s)
I20260812 11:20:11.826058 3934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:11.833060 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 3934
I20260812 11:20:11.833187 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:11.833498 31742 external_mini_cluster.cc:1428] Setting key a6e840d12af7941fbfa39cf765f2483c
I20260812 11:20:11.834357 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.827701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:58367 (local address 127.30.255.190:42239)
0812 11:20:11.827889 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:11.827892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:11.828794 (+ 902us) server_negotiation.cc:407] Connection header received
0812 11:20:11.830072 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:11.830075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:11.830127 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:11.830194 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:11.830950 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.831483 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.832558 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.832764 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.832878 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.833916 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:11.833921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:11.833923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:11.833936 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:11.833960 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:11.833969 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:11.834026 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:11.834186 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 11:20:11.834243 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 11:20:11.835230 4185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:11.828051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:42239 (local address 127.30.255.130:58367)
0812 11:20:11.828656 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:11.828692 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:11.829894 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:11.830330 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:11.830337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:11.830415 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:11.830845 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.830857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.831587 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.831590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.832470 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:11.832481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:11.832907 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:11.832910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:11.833066 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:11.833634 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:11.833636 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:11.833697 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:11.833773 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:11.834047 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:11.834057 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:11.834062 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 11:20:11.834167 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:20:11.834415 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":112,"threads_started":1}
I20260812 11:20:11.835498 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--builtin_ntp_servers=127.30.255.148:36115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:11.836279 4182 heartbeater.cc:347] Connected to a master server at 127.30.255.190:42239
I20260812 11:20:11.836484 4182 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:11.836941 4182 heartbeater.cc:511] Master 127.30.255.190:42239 requested a full tablet report, sending...
I20260812 11:20:11.837870 3541 ts_manager.cc:194] Registered new tserver with Master: a7c923c15e904df99d163e299d263fdb (127.30.255.130:40009)
I20260812 11:20:11.838392 3541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.130:58367
W20260812 11:20:11.945694 4186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:11.945909 4186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:11.945968 4186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:11.949899 4186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:11.949944 4186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:11.950013 4186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:11.954543 4186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:42239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:11.955453 4186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:11.956645 4186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:11.965303 4197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.965445 4186 server_base.cc:1034] running on GCE node
W20260812 11:20:11.965342 4202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:11.965305 4199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:11.965910 4186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:11.966480 4186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:11.967634 4186 hybrid_clock.cc:648] HybridClock initialized: now 1786533611967618 us; error 31 us; skew 500 ppm
I20260812 11:20:11.969267 4186 webserver.cc:460] Webserver started at http://127.30.255.131:37293/ using document root <none> and password file <none>
I20260812 11:20:11.969775 4186 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:11.969846 4186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:11.970021 4186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:11.971909 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "239789c0fc3c43bd8029eee110d6f6fe"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "805e25169b50c827c2c3a4d73435511e"
server_key_iv: "c2c83ad4666d8302db6e5552dd796689"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.972420 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "239789c0fc3c43bd8029eee110d6f6fe"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "805e25169b50c827c2c3a4d73435511e"
server_key_iv: "c2c83ad4666d8302db6e5552dd796689"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.975700 4186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:11.977895 4224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.979022 4186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:11.979131 4186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "239789c0fc3c43bd8029eee110d6f6fe"
format_stamp: "Formatted at 2026-08-12 11:20:11 on dist-test-slave-gjrz"
server_key: "805e25169b50c827c2c3a4d73435511e"
server_key_iv: "c2c83ad4666d8302db6e5552dd796689"
server_key_version: "encryptionkey@0"
I20260812 11:20:11.979218 4186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:11.991099 4186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:11.991683 4186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:11.991834 4186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:11.992348 4186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:11.993165 4186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:11.993240 4186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:11.993292 4186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:11.993316 4186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:12.001233 4186 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:46175
I20260812 11:20:12.001264 4437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:46175 every 8 connection(s)
I20260812 11:20:12.001993 4186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:20:12.002318 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4186
I20260812 11:20:12.002414 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:12.002712 31742 external_mini_cluster.cc:1428] Setting key aa740f3cb17ae20de8e98efd1e1f7b34
I20260812 11:20:12.010677 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.003809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35997 (local address 127.30.255.190:42239)
0812 11:20:12.003915 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.003919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.005040 (+ 1121us) server_negotiation.cc:407] Connection header received
0812 11:20:12.006336 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.006339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.006393 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.006462 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:12.007347 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.007864 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.008995 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.009218 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.009302 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.010239 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:12.010243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:12.010245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:12.010258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:12.010279 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:12.010288 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:12.010346 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.010479 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.010540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 11:20:12.011425 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.004250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:42239 (local address 127.30.255.131:35997)
0812 11:20:12.004911 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.004947 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.006152 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.006627 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.006633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.006710 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:12.007212 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.007226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.007978 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.007981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.008873 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.008884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.009347 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.009352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.009511 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.010008 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:12.010009 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:12.010045 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:12.010103 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:12.010359 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:12.010368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:12.010373 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.010478 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.010723 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260812 11:20:12.012351 4438 heartbeater.cc:347] Connected to a master server at 127.30.255.190:42239
I20260812 11:20:12.012548 4438 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:12.013024 4438 heartbeater.cc:511] Master 127.30.255.190:42239 requested a full tablet report, sending...
I20260812 11:20:12.014061 3541 ts_manager.cc:194] Registered new tserver with Master: 239789c0fc3c43bd8029eee110d6f6fe (127.30.255.131:46175)
I20260812 11:20:12.014592 3541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.131:35997
I20260812 11:20:12.016310 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:12.021302 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.018037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.30.255.190:42239)
0812 11:20:12.018159 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.018163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.018326 (+ 163us) server_negotiation.cc:407] Connection header received
0812 11:20:12.018402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.018404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.018440 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.018498 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:12.018693 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.019160 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.019942 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.020144 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.020214 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.020917 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:12.020920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:12.020922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:12.020933 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:12.020950 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:12.020957 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:12.021050 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.021141 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.021174 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:20:12.023769 3541 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:12.026533 3541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:12.039947 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.034313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:35937 (local address 127.0.0.1:53504)
0812 11:20:12.034569 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.034581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.034676 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.036006 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.036009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.036029 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:12.036109 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.036115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.037101 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.037105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.038391 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.038402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.039417 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.039420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.039538 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.039581 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.039691 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.039806 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":48,"thread_start_us":95,"threads_started":1}
I20260812 11:20:12.039999 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.035124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53504 (local address 127.30.255.129:35937)
0812 11:20:12.035563 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.035575 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.035599 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:20:12.035688 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.035694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.035871 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.036025 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:12.036211 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.036998 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.038519 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.039293 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.039407 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.039567 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.039736 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.039839 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":103,"threads_started":1}
I20260812 11:20:12.040385 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.034133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.30.255.130:40009)
0812 11:20:12.035625 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.035635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.035655 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:20:12.035718 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.035722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.035872 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.036025 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:12.036430 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.037547 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.038820 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.039404 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.039548 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.039716 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.039878 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.039977 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1400,"thread_start_us":112,"threads_started":1}
I20260812 11:20:12.041241 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.034669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:46175 (local address 127.0.0.1:44516)
0812 11:20:12.034906 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.034922 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.035007 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.035652 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.035658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.035680 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:12.035798 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.035804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.037041 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.037045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.038366 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.038376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.040014 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.040018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.040949 (+ 931us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.040970 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.041042 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.041107 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":103,"thread_start_us":97,"threads_started":1}
I20260812 11:20:12.041749 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.034789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44516 (local address 127.30.255.131:46175)
0812 11:20:12.035161 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.035171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.035194 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:20:12.035259 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.035291 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.035489 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.035649 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:12.035901 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.036921 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.038485 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.039288 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.039407 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.039567 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.041517 (+ 1950us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.041611 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":93,"threads_started":1}
I20260812 11:20:12.042006 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.033973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:40009 (local address 127.0.0.1:43036)
0812 11:20:12.034270 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.034309 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.034420 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.036196 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.036199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.036217 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:12.036300 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.036306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.037681 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.037684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.038713 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.038721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.039649 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.039652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.039739 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.039763 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.039847 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.039916 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":31,"thread_start_us":58,"threads_started":1}
I20260812 11:20:12.043035 3796 tablet_service.cc:1467] Processing CreateTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 (DEFAULT_TABLE table=test-table [id=d712cbb0d4244b6a9166de7f68a275d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:12.044238 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:12.044167 4313 tablet_service.cc:1467] Processing CreateTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 (DEFAULT_TABLE table=test-table [id=d712cbb0d4244b6a9166de7f68a275d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:12.045070 4313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:12.044780 4055 tablet_service.cc:1467] Processing CreateTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 (DEFAULT_TABLE table=test-table [id=d712cbb0d4244b6a9166de7f68a275d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:12.045742 4055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:12.050482 4473 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Bootstrap starting.
I20260812 11:20:12.052469 4473 tablet_bootstrap.cc:654] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:12.052906 4475 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Bootstrap starting.
I20260812 11:20:12.053318 4473 log.cc:824] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:12.054417 4476 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Bootstrap starting.
I20260812 11:20:12.055163 4475 tablet_bootstrap.cc:654] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:12.055991 4475 log.cc:824] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:12.056595 4473 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: No bootstrap required, opened a new log
I20260812 11:20:12.056789 4473 ts_tablet_manager.cc:1397] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 11:20:12.057042 4476 tablet_bootstrap.cc:654] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:12.057663 4476 log.cc:824] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:12.057988 4475 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: No bootstrap required, opened a new log
I20260812 11:20:12.058192 4475 ts_tablet_manager.cc:1397] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:20:12.059175 4476 tablet_bootstrap.cc:492] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: No bootstrap required, opened a new log
I20260812 11:20:12.059367 4476 ts_tablet_manager.cc:1397] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:20:12.060918 4473 raft_consensus.cc:348] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.061185 4473 raft_consensus.cc:374] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:12.061244 4473 raft_consensus.cc:729] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239789c0fc3c43bd8029eee110d6f6fe, State: Initialized, Role: FOLLOWER
I20260812 11:20:12.061684 4473 consensus_queue.cc:260] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.062342 4438 heartbeater.cc:503] Master 127.30.255.190:42239 was elected leader, sending a full tablet report...
I20260812 11:20:12.062273 4475 raft_consensus.cc:348] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.062498 4475 raft_consensus.cc:374] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:12.062552 4475 raft_consensus.cc:729] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e8d73209c74d5b89ef3e08a9c04337, State: Initialized, Role: FOLLOWER
I20260812 11:20:12.062935 4473 ts_tablet_manager.cc:1428] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:12.062949 4475 consensus_queue.cc:260] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.062906 4476 raft_consensus.cc:348] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.063133 4476 raft_consensus.cc:374] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:12.063181 4476 raft_consensus.cc:729] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c923c15e904df99d163e299d263fdb, State: Initialized, Role: FOLLOWER
I20260812 11:20:12.063637 4476 consensus_queue.cc:260] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.063743 3929 heartbeater.cc:503] Master 127.30.255.190:42239 was elected leader, sending a full tablet report...
I20260812 11:20:12.063865 4475 ts_tablet_manager.cc:1428] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:20:12.064539 4476 ts_tablet_manager.cc:1428] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:20:12.064865 4182 heartbeater.cc:503] Master 127.30.255.190:42239 was elected leader, sending a full tablet report...
W20260812 11:20:12.076828 4184 tablet.cc:2369] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:20:12.142948 3930 tablet.cc:2369] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:12.198549 4483 raft_consensus.cc:482] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:12.198735 4483 raft_consensus.cc:504] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.199851 4483 leader_election.cc:288] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7c923c15e904df99d163e299d263fdb (127.30.255.130:40009), 239789c0fc3c43bd8029eee110d6f6fe (127.30.255.131:46175)
I20260812 11:20:12.203503 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.200150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:46741 (local address 127.30.255.130:40009)
0812 11:20:12.200375 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.200380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.200499 (+ 119us) server_negotiation.cc:407] Connection header received
0812 11:20:12.200625 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.200628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.200672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.200732 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:12.201022 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.201513 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.202612 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.202983 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.203085 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.203135 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.203311 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.203379 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260812 11:20:12.203549 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.200146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:50747 (local address 127.30.255.131:46175)
0812 11:20:12.200271 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.200275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.200415 (+ 140us) server_negotiation.cc:407] Connection header received
0812 11:20:12.200551 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.200553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.200598 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.200669 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:12.200967 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.201464 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.202615 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.203014 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.203122 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.203171 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.203363 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.203432 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:20:12.203572 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.200057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:40009 (local address 127.30.255.129:46741)
0812 11:20:12.200421 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.200432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.200526 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.200749 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.200752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.200767 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:12.200915 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.200920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.201621 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.201624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.202494 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.202504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.203089 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.203092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.203184 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.203204 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.203306 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.203374 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":121,"thread_start_us":64,"threads_started":1}
I20260812 11:20:12.203576 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.200052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:46175 (local address 127.30.255.129:50747)
0812 11:20:12.200330 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.200342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:12.200449 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:12.200690 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:12.200693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:12.200709 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:12.200854 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.200860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.201569 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.201572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.202487 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:12.202497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.203144 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.203147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:12.203234 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.203251 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:20:12.203348 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:20:12.203415 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":81,"thread_start_us":88,"threads_started":1}
I20260812 11:20:12.204310 4354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91a7ce5e8ba4e8ebeefb891fe79bea2" candidate_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239789c0fc3c43bd8029eee110d6f6fe" is_pre_election: true
I20260812 11:20:12.204365 4096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91a7ce5e8ba4e8ebeefb891fe79bea2" candidate_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7c923c15e904df99d163e299d263fdb" is_pre_election: true
I20260812 11:20:12.204579 4354 raft_consensus.cc:2454] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e8d73209c74d5b89ef3e08a9c04337 in term 0.
I20260812 11:20:12.204643 4096 raft_consensus.cc:2454] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e8d73209c74d5b89ef3e08a9c04337 in term 0.
I20260812 11:20:12.205093 3712 leader_election.cc:302] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 239789c0fc3c43bd8029eee110d6f6fe, a7c923c15e904df99d163e299d263fdb, d6e8d73209c74d5b89ef3e08a9c04337; no voters:
I20260812 11:20:12.205346 4483 raft_consensus.cc:2781] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:12.205425 4483 raft_consensus.cc:482] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:12.205454 4483 raft_consensus.cc:3037] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:12.206473 4483 raft_consensus.cc:504] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.206806 4483 leader_election.cc:288] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [CANDIDATE]: Term 1 election: Requested vote from peers a7c923c15e904df99d163e299d263fdb (127.30.255.130:40009), 239789c0fc3c43bd8029eee110d6f6fe (127.30.255.131:46175)
I20260812 11:20:12.207098 4096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91a7ce5e8ba4e8ebeefb891fe79bea2" candidate_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7c923c15e904df99d163e299d263fdb"
I20260812 11:20:12.207118 4354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91a7ce5e8ba4e8ebeefb891fe79bea2" candidate_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239789c0fc3c43bd8029eee110d6f6fe"
I20260812 11:20:12.207216 4096 raft_consensus.cc:3037] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:12.207247 4354 raft_consensus.cc:3037] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:12.208266 4096 raft_consensus.cc:2454] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e8d73209c74d5b89ef3e08a9c04337 in term 1.
I20260812 11:20:12.208307 4354 raft_consensus.cc:2454] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e8d73209c74d5b89ef3e08a9c04337 in term 1.
I20260812 11:20:12.208578 3711 leader_election.cc:302] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7c923c15e904df99d163e299d263fdb, d6e8d73209c74d5b89ef3e08a9c04337; no voters:
I20260812 11:20:12.208778 4483 raft_consensus.cc:2781] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:12.208940 4483 raft_consensus.cc:686] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [term 1 LEADER]: Becoming Leader. State: Replica: d6e8d73209c74d5b89ef3e08a9c04337, State: Running, Role: LEADER
I20260812 11:20:12.209231 4483 consensus_queue.cc:237] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } }
I20260812 11:20:12.212218 3529 catalog_manager.cc:5495] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6e8d73209c74d5b89ef3e08a9c04337 (127.30.255.129). New cstate: current_term: 1 leader_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e8d73209c74d5b89ef3e08a9c04337" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 35937 } health_report { overall_health: HEALTHY } } }
I20260812 11:20:12.229511 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.226576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.30.255.129:35937)
0812 11:20:12.226682 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:12.226685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:12.226697 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:12.226798 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:12.226800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:12.226848 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:12.226926 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:12.227111 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.227612 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.228387 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:12.228582 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:12.228651 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:12.228868 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:12.229226 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:12.229330 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:12.229383 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 11:20:12.229413 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:20:12.234795 4354 raft_consensus.cc:1261] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 1 FOLLOWER]: Refusing update from remote peer d6e8d73209c74d5b89ef3e08a9c04337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:12.234794 4096 raft_consensus.cc:1261] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 1 FOLLOWER]: Refusing update from remote peer d6e8d73209c74d5b89ef3e08a9c04337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:12.235473 4490 consensus_queue.cc:1035] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239789c0fc3c43bd8029eee110d6f6fe" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:12.235643 4483 consensus_queue.cc:1035] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P d6e8d73209c74d5b89ef3e08a9c04337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7c923c15e904df99d163e299d263fdb" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 40009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:12.241835 4493 mvcc.cc:205] Tried to move back new op lower bound from 7317641675707592704 to 7317641675611332608. Current Snapshot: MvccSnapshot[applied={T|T < 7317641675707592704}]
I20260812 11:20:12.242399 4494 mvcc.cc:205] Tried to move back new op lower bound from 7317641675707592704 to 7317641675611332608. Current Snapshot: MvccSnapshot[applied={T|T < 7317641675707592704}]
I20260812 11:20:12.245852 4502 mvcc.cc:205] Tried to move back new op lower bound from 7317641675707592704 to 7317641675611332608. Current Snapshot: MvccSnapshot[applied={T|T < 7317641675707592704 or (T in {7317641675707592704})}]
I20260812 11:20:12.250488 3529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:12.250638 3529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:12.252784 3529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 648c250bbf454b2b818b957f7734578d: Sending DeleteTablet for 3 replicas of tablet e91a7ce5e8ba4e8ebeefb891fe79bea2
I20260812 11:20:12.253453 4055 tablet_service.cc:1514] Processing DeleteTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:12 UTC) from {username='slave'} at 127.0.0.1:43036
I20260812 11:20:12.253629 3796 tablet_service.cc:1514] Processing DeleteTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:12 UTC) from {username='slave'} at 127.0.0.1:53504
I20260812 11:20:12.254139 4313 tablet_service.cc:1514] Processing DeleteTablet for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:12 UTC) from {username='slave'} at 127.0.0.1:44516
I20260812 11:20:12.254506 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3672
I20260812 11:20:12.262565 4514 tablet_replica.cc:330] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: stopping tablet replica
I20260812 11:20:12.262823 4518 tablet_replica.cc:330] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: stopping tablet replica
I20260812 11:20:12.262876 4514 raft_consensus.cc:2229] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:12.263098 4518 raft_consensus.cc:2229] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:12.263178 4514 raft_consensus.cc:2258] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:12.263392 4518 raft_consensus.cc:2258] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:12.264698 4514 ts_tablet_manager.cc:1905] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:12.264698 4518 ts_tablet_manager.cc:1905] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:12.267302 4514 ts_tablet_manager.cc:1918] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:12.267298 4518 ts_tablet_manager.cc:1918] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:12.267395 4518 log.cc:1196] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/e91a7ce5e8ba4e8ebeefb891fe79bea2
I20260812 11:20:12.267395 4514 log.cc:1196] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/e91a7ce5e8ba4e8ebeefb891fe79bea2
I20260812 11:20:12.267719 4514 ts_tablet_manager.cc:1939] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P a7c923c15e904df99d163e299d263fdb: Deleting consensus metadata
I20260812 11:20:12.267719 4518 ts_tablet_manager.cc:1939] T e91a7ce5e8ba4e8ebeefb891fe79bea2 P 239789c0fc3c43bd8029eee110d6f6fe: Deleting consensus metadata
W20260812 11:20:12.268708 3499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:35937 (error 108)
I20260812 11:20:12.268833 3503 catalog_manager.cc:4841] TS a7c923c15e904df99d163e299d263fdb (127.30.255.130:40009): tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 (table test-table [id=d712cbb0d4244b6a9166de7f68a275d5]) successfully deleted
I20260812 11:20:12.269138 3508 catalog_manager.cc:4841] TS 239789c0fc3c43bd8029eee110d6f6fe (127.30.255.131:46175): tablet e91a7ce5e8ba4e8ebeefb891fe79bea2 (table test-table [id=d712cbb0d4244b6a9166de7f68a275d5]) successfully deleted
I20260812 11:20:12.269213 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3934
I20260812 11:20:12.291112 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:12.269434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:35937 (local address 127.0.0.1:53512)
0812 11:20:12.290966 (+ 21532us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:12.291037 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:35937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21439}
W20260812 11:20:12.291311 3499 catalog_manager.cc:4584] TS d6e8d73209c74d5b89ef3e08a9c04337 (127.30.255.129:35937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e91a7ce5e8ba4e8ebeefb891fe79bea2: Network error: Client connection negotiation failed: client connection to 127.30.255.129:35937: connect: Connection refused (error 111)
I20260812 11:20:12.297246 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4186
I20260812 11:20:12.306989 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 3472
2026-08-12T11:20:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1149 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:12 dist-test-slave-gjrz krb5kdc[4524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:12 dist-test-slave-gjrz krb5kdc[4524](info): set up 2 sockets
Aug 12 11:20:12 dist-test-slave-gjrz krb5kdc[4524](info): commencing operation
krb5kdc: starting...
W20260812 11:20:13.507877 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:13 dist-test-slave-gjrz krb5kdc[4524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:13.621351 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44773
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:13.728343 4548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:13.728576 4548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:13.728617 4548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:13.732604 4548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:13.732652 4548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:13.732672 4548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:13.732702 4548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:13.732728 4548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:13.737741 4548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:13.738653 4548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:13.739638 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:13.747536 4559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:13.747547 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:13.747622 4548 server_base.cc:1034] running on GCE node
W20260812 11:20:13.747613 4558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:13.748301 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:13.749275 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:13.750423 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786533613750409 us; error 25 us; skew 500 ppm
Aug 12 11:20:13 dist-test-slave-gjrz krb5kdc[4524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533613, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:13.752918 4548 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:13.753726 4548 webserver.cc:460] Webserver started at http://127.30.255.190:46763/ using document root <none> and password file <none>
I20260812 11:20:13.754202 4548 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:13.754257 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:13.754415 4548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:13.756299 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "133912cac07b4e16b9c298011b7285b7"
format_stamp: "Formatted at 2026-08-12 11:20:13 on dist-test-slave-gjrz"
server_key: "2134535832b45594db96506426dd5c1f"
server_key_iv: "52b02c904d741c132fe5b5753ac4be09"
server_key_version: "encryptionkey@0"
I20260812 11:20:13.756788 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "133912cac07b4e16b9c298011b7285b7"
format_stamp: "Formatted at 2026-08-12 11:20:13 on dist-test-slave-gjrz"
server_key: "2134535832b45594db96506426dd5c1f"
server_key_iv: "52b02c904d741c132fe5b5753ac4be09"
server_key_version: "encryptionkey@0"
I20260812 11:20:13.760150 4548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:13.762259 4580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:13.763417 4548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:13.763515 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "133912cac07b4e16b9c298011b7285b7"
format_stamp: "Formatted at 2026-08-12 11:20:13 on dist-test-slave-gjrz"
server_key: "2134535832b45594db96506426dd5c1f"
server_key_iv: "52b02c904d741c132fe5b5753ac4be09"
server_key_version: "encryptionkey@0"
I20260812 11:20:13.763604 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:13.794922 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:13.797382 4548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:13.797523 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:13.803910 4548 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:44773
I20260812 11:20:13.803925 4695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:44773 every 8 connection(s)
I20260812 11:20:13.804850 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:13.806360 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4548
I20260812 11:20:13.806511 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:13.806838 31742 external_mini_cluster.cc:1428] Setting key 0b1e7972189e7fbef1bc7a4e0cf77635
I20260812 11:20:13.808003 4697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:13.814234 4697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: Bootstrap starting.
Aug 12 11:20:13 dist-test-slave-gjrz krb5kdc[4524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:13.816797 4697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:13.817488 4697 log.cc:824] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:13.819573 4697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: No bootstrap required, opened a new log
I20260812 11:20:13.820248 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:13.808156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.30.255.190:44773)
0812 11:20:13.808528 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:13.808550 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:13.808586 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:20:13.809200 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:13.809215 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:13.809515 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:13.809785 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:13.810684 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:13.811702 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:13.812833 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:13.813269 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:13.813374 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:13.815988 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:13.815994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:13.816009 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:13.816039 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:13.817592 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:13.818061 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:13.818066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:13.818072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:13.818212 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:13.818479 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:13.818481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:13.818482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:13.818827 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:13.818958 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:13.819255 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 11:20:13.819551 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
I20260812 11:20:13.823156 4697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } }
I20260812 11:20:13.823396 4697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:13.823457 4697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 133912cac07b4e16b9c298011b7285b7, State: Initialized, Role: FOLLOWER
I20260812 11:20:13.823868 4697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } }
I20260812 11:20:13.823998 4697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:13.824062 4697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:13.824157 4697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:13.824931 4697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } }
I20260812 11:20:13.825233 4697 leader_election.cc:302] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 133912cac07b4e16b9c298011b7285b7; no voters:
I20260812 11:20:13.825495 4697 leader_election.cc:288] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:13.825606 4703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:13.825827 4703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [term 1 LEADER]: Becoming Leader. State: Replica: 133912cac07b4e16b9c298011b7285b7, State: Running, Role: LEADER
I20260812 11:20:13.826153 4703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } }
I20260812 11:20:13.826632 4697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:13.827747 4708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 133912cac07b4e16b9c298011b7285b7. Latest consensus state: current_term: 1 leader_uuid: "133912cac07b4e16b9c298011b7285b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } } }
I20260812 11:20:13.827747 4705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "133912cac07b4e16b9c298011b7285b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133912cac07b4e16b9c298011b7285b7" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 44773 } } }
I20260812 11:20:13.827898 4708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:13.827898 4705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:13.828287 4721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:13.831067 4721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:13.835809 4721 catalog_manager.cc:1347] Generated new cluster ID: 89151c32d7fe48a8bee6bbf91521816f
I20260812 11:20:13.835916 4721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:13.850734 4721 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:13.851557 4721 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:13.858642 4721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: Generated new TSK 0
I20260812 11:20:13.859205 4721 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.255.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
Entry for principal kudu/127.30.255.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab.
I20260812 11:20:13.893790 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:14.000218 4744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:14.000435 4744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:14.000481 4744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:14.004318 4744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:14.004364 4744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:14.004417 4744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:14.009318 4744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.129.keytab
--principal=kudu/127.30.255.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:14.010174 4744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:14.011049 4744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:14.019822 4752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.019809 4750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.020025 4744 server_base.cc:1034] running on GCE node
W20260812 11:20:14.019845 4755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.020439 4744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:14.021121 4744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:14.022284 4744 hybrid_clock.cc:648] HybridClock initialized: now 1786533614022257 us; error 37 us; skew 500 ppm
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:14.024870 4744 init.cc:377] Logged in from keytab as kudu/127.30.255.129@KRBTEST.COM (short username kudu)
I20260812 11:20:14.025717 4744 webserver.cc:460] Webserver started at http://127.30.255.129:33811/ using document root <none> and password file <none>
I20260812 11:20:14.026216 4744 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:14.026276 4744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:14.026432 4744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:14.028436 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "eff5cd9a436f4499b2d9a955c58b9e20"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "d624a1a151c351b94d7d1e1b47daeff2"
server_key_iv: "f6a8d1e380b58060cdcecbd095c28dd2"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.028954 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "eff5cd9a436f4499b2d9a955c58b9e20"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "d624a1a151c351b94d7d1e1b47daeff2"
server_key_iv: "f6a8d1e380b58060cdcecbd095c28dd2"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.032622 4744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:14.034821 4771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.036011 4744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:14.036115 4744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "eff5cd9a436f4499b2d9a955c58b9e20"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "d624a1a151c351b94d7d1e1b47daeff2"
server_key_iv: "f6a8d1e380b58060cdcecbd095c28dd2"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.036202 4744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:14.063896 4744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:14.066608 4744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:14.066766 4744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:14.067312 4744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:14.068068 4744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:14.068121 4744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.068157 4744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:14.068177 4744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.076318 4744 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:34355
I20260812 11:20:14.076411 4998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:34355 every 8 connection(s)
I20260812 11:20:14.077138 4744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:20:14.079159 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 4744
I20260812 11:20:14.079303 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:14.079595 31742 external_mini_cluster.cc:1428] Setting key fc0e8b8b7be97b93675734316df0c5d8
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.129@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.130@KRBTEST.COM; defaulting to no policy
I20260812 11:20:14.090679 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.078770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:40701 (local address 127.30.255.190:44773)
0812 11:20:14.078911 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.078914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.079743 (+ 829us) server_negotiation.cc:407] Connection header received
0812 11:20:14.081165 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.081168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.081218 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.081310 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:14.082372 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.083081 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.084612 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.084945 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.085062 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.087209 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:14.087213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:14.087216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:14.087245 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:14.088829 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.089290 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.089294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.089296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.089384 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.089681 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.089684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.089686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.089905 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:14.090021 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.090448 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.090557 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:20:14.091445 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.079116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44773 (local address 127.30.255.129:40701)
0812 11:20:14.079598 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.079649 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.080964 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.081441 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.081448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.081765 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:14.082226 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.082239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.083199 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.083202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.084494 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.084505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.085084 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.085087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.085222 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.085718 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:14.085733 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:14.087046 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:14.088939 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.088943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.088954 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.089192 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.089478 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.089480 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.089481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.089597 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.089992 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:14.089996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:14.090236 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.090381 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.090616 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":161,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.130@KRBTEST.COM" created.
I20260812 11:20:14.092614 5002 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44773
I20260812 11:20:14.092819 5002 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:14.093266 5002 heartbeater.cc:511] Master 127.30.255.190:44773 requested a full tablet report, sending...
I20260812 11:20:14.094707 4619 ts_manager.cc:194] Registered new tserver with Master: eff5cd9a436f4499b2d9a955c58b9e20 (127.30.255.129:34355)
I20260812 11:20:14.095832 4619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.129@KRBTEST.COM'} at 127.30.255.129:40701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
Entry for principal kudu/127.30.255.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab.
I20260812 11:20:14.106140 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:14.213441 5010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:14.213656 5010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:14.213697 5010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:14.217597 5010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:14.217643 5010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:14.217705 5010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:14.222664 5010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.130.keytab
--principal=kudu/127.30.255.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:14.223538 5010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:14.224422 5010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:14.233399 5019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.233554 5027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.233407 5020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.233593 5010 server_base.cc:1034] running on GCE node
I20260812 11:20:14.234076 5010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:14.234618 5010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:14.235771 5010 hybrid_clock.cc:648] HybridClock initialized: now 1786533614235756 us; error 25 us; skew 500 ppm
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:14.238262 5010 init.cc:377] Logged in from keytab as kudu/127.30.255.130@KRBTEST.COM (short username kudu)
I20260812 11:20:14.239073 5010 webserver.cc:460] Webserver started at http://127.30.255.130:39169/ using document root <none> and password file <none>
I20260812 11:20:14.239598 5010 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:14.239665 5010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:14.239856 5010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:14.241778 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "72ab7537b29640508b6691d7ab9ad2c2"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "7d3fa8c4d80d4b5ef6e999a497eb852b"
server_key_iv: "4b4ce46edb19bd84c854f99d0e980750"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.242298 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "72ab7537b29640508b6691d7ab9ad2c2"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "7d3fa8c4d80d4b5ef6e999a497eb852b"
server_key_iv: "4b4ce46edb19bd84c854f99d0e980750"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.245654 5010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:14.247812 5042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.248865 5010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:14.248986 5010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "72ab7537b29640508b6691d7ab9ad2c2"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "7d3fa8c4d80d4b5ef6e999a497eb852b"
server_key_iv: "4b4ce46edb19bd84c854f99d0e980750"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.249073 5010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:14.266044 5010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:14.268564 5010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:14.268723 5010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:14.269238 5010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:14.269987 5010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:14.270035 5010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.270085 5010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:14.270114 5010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.278312 5010 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:37425
I20260812 11:20:14.278338 5244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:37425 every 8 connection(s)
I20260812 11:20:14.279103 5010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:14.281708 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5010
I20260812 11:20:14.281837 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:14.282135 31742 external_mini_cluster.cc:1428] Setting key 571582eef2276174dcc3b38ebdc1af01
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.130@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.131@KRBTEST.COM; defaulting to no policy
I20260812 11:20:14.292025 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.280731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:55651 (local address 127.30.255.190:44773)
0812 11:20:14.280862 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.280866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.281657 (+ 791us) server_negotiation.cc:407] Connection header received
0812 11:20:14.283051 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.283055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.283108 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.283182 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:14.284580 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.285100 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.286333 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.286559 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.286646 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.288863 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:14.288866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:14.288868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:14.288889 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:14.290156 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.290618 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.290621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.290622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.290697 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.290996 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.290999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.291001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.291176 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:14.291280 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.291774 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.291904 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:20:14.292738 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.281107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44773 (local address 127.30.255.130:55651)
0812 11:20:14.281516 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.281563 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.282850 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.283378 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.283385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.283818 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:14.284432 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.284450 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.285219 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.285222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.286213 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.286223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.286691 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.286695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.286848 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.287359 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:14.287375 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:14.288710 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:14.290259 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.290265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.290279 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.290518 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.290779 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.290782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.290784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.290910 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.291294 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:14.291300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:14.291525 (+ 225us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.291700 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.291934 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260812 11:20:14.293781 5246 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44773
I20260812 11:20:14.293987 5246 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.131@KRBTEST.COM" created.
I20260812 11:20:14.294488 5246 heartbeater.cc:511] Master 127.30.255.190:44773 requested a full tablet report, sending...
I20260812 11:20:14.295482 4619 ts_manager.cc:194] Registered new tserver with Master: 72ab7537b29640508b6691d7ab9ad2c2 (127.30.255.130:37425)
I20260812 11:20:14.296056 4619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.130@KRBTEST.COM'} at 127.30.255.130:55651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
Entry for principal kudu/127.30.255.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab.
I20260812 11:20:14.307566 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:14.413386 5257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:14.413610 5257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:14.413666 5257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:14.417512 5257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:14.417564 5257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:14.417641 5257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:14.422541 5257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.131.keytab
--principal=kudu/127.30.255.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:44773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:14.423415 5257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:14.424288 5257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:14.433176 5274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.433184 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.433264 5257 server_base.cc:1034] running on GCE node
W20260812 11:20:14.433175 5277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.433782 5257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:14.434335 5257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:14.435482 5257 hybrid_clock.cc:648] HybridClock initialized: now 1786533614435464 us; error 26 us; skew 500 ppm
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:14.437906 5257 init.cc:377] Logged in from keytab as kudu/127.30.255.131@KRBTEST.COM (short username kudu)
I20260812 11:20:14.438674 5257 webserver.cc:460] Webserver started at http://127.30.255.131:39505/ using document root <none> and password file <none>
I20260812 11:20:14.439141 5257 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:14.439208 5257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:14.439488 5257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:14.441361 5257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "12ee408968d945a891fffb8169186c55"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "72dbb30b1bca23edfdb6548c0cb4db56"
server_key_iv: "4cf5c8128b323de1594d22c2723a2198"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.441854 5257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "12ee408968d945a891fffb8169186c55"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "72dbb30b1bca23edfdb6548c0cb4db56"
server_key_iv: "4cf5c8128b323de1594d22c2723a2198"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.445225 5257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:14.447418 5291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.448505 5257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:14.448606 5257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "12ee408968d945a891fffb8169186c55"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "72dbb30b1bca23edfdb6548c0cb4db56"
server_key_iv: "4cf5c8128b323de1594d22c2723a2198"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.448683 5257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:14.467725 5257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:14.470180 5257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:14.470315 5257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:14.470796 5257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:14.471547 5257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:14.471592 5257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.471627 5257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:14.471647 5257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.479578 5257 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:40903
I20260812 11:20:14.479599 5509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:40903 every 8 connection(s)
I20260812 11:20:14.480350 5257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:20:14.482458 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5257
I20260812 11:20:14.482583 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:14.482913 31742 external_mini_cluster.cc:1428] Setting key 58f1992131e009c7d79c7ea6269ef17c
Aug 12 11:20:14 dist-test-slave-gjrz krb5kdc[4524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.131@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:14.492905 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.481935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:35175 (local address 127.30.255.190:44773)
0812 11:20:14.482087 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.482092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.482935 (+ 843us) server_negotiation.cc:407] Connection header received
0812 11:20:14.484546 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.484550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.484605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.484683 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:14.485843 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.486362 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.487513 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.487736 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.487823 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.490012 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:14.490016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:14.490018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:14.490042 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:14.491339 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.491773 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.491775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.491776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.491843 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.492085 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.492087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.492088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.492244 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:14.492314 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.492699 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.492790 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:20:14.493592 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.482289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:44773 (local address 127.30.255.131:35175)
0812 11:20:14.482771 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.482826 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.484359 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.484878 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.484887 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.485260 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:14.485706 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.485719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.486480 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.486484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.487399 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.487409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.487874 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.487878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.488035 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.488511 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:14.488527 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:14.489863 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:14.491446 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.491450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.491460 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.491701 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.491905 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:14.491907 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:14.491908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:14.492023 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:14.492312 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:14.492316 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:14.492516 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.492667 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.492886 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260812 11:20:14.494514 5511 heartbeater.cc:347] Connected to a master server at 127.30.255.190:44773
I20260812 11:20:14.494704 5511 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:14.495113 5511 heartbeater.cc:511] Master 127.30.255.190:44773 requested a full tablet report, sending...
I20260812 11:20:14.496141 4619 ts_manager.cc:194] Registered new tserver with Master: 12ee408968d945a891fffb8169186c55 (127.30.255.131:40903)
I20260812 11:20:14.496634 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:14.496663 4619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.255.131@KRBTEST.COM'} at 127.30.255.131:35175
I20260812 11:20:14.504726 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.498672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.30.255.190:44773)
0812 11:20:14.498780 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.498783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.498943 (+ 160us) server_negotiation.cc:407] Connection header received
0812 11:20:14.499014 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.499016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.499049 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.499110 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:14.499735 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.500211 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.500939 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.501137 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.501201 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.502264 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:14.502267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:14.502269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:14.502288 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:14.503556 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.503863 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.503865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.503867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.503931 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:14.504113 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:14.504115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:14.504116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:14.504304 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:14.504401 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.504545 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.504625 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:20:14.507057 4619 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:14.509878 4619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:14.522570 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.517436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:40903 (local address 127.0.0.1:59178)
0812 11:20:14.518122 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.518137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.518251 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.518921 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.518927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.518947 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:14.519071 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.519077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.520157 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.520160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.521102 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.521111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.522054 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.522058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.522174 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.522210 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.522319 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.522435 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"spinlock_wait_cycles":1920,"thread_start_us":88,"threads_started":1}
I20260812 11:20:14.522581 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.517918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.30.255.131:40903)
0812 11:20:14.518508 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.518519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.518551 (+ 32us) server_negotiation.cc:407] Connection header received
0812 11:20:14.518614 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.518620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.518782 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.518912 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:14.519180 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.520051 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.521201 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.521940 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.522062 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.522176 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.522357 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.522442 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":89,"threads_started":1}
I20260812 11:20:14.523213 5527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.517274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37425 (local address 127.0.0.1:47218)
0812 11:20:14.518031 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.518078 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.518255 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.518950 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.518952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.518977 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:14.519093 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.519099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.520520 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.520523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.521616 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.521626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.522821 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.522825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.522924 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.522947 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.523015 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.523090 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"spinlock_wait_cycles":384,"thread_start_us":71,"threads_started":1}
I20260812 11:20:14.523645 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.517404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.30.255.130:37425)
0812 11:20:14.517810 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.517821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.518179 (+ 358us) server_negotiation.cc:407] Connection header received
0812 11:20:14.518428 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.518434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.518640 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.518772 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:14.519232 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.520369 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.521745 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.522520 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.522635 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.522755 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.523458 (+ 703us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.523541 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":111,"threads_started":1}
I20260812 11:20:14.525462 5378 tablet_service.cc:1467] Processing CreateTablet for tablet 711733a7d30d48fc8a924ee54234b980 (DEFAULT_TABLE table=test-table [id=f5c9f53c9d9745ff91b24fce8c938e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:14.526638 5378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711733a7d30d48fc8a924ee54234b980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:14.526288 5134 tablet_service.cc:1467] Processing CreateTablet for tablet 711733a7d30d48fc8a924ee54234b980 (DEFAULT_TABLE table=test-table [id=f5c9f53c9d9745ff91b24fce8c938e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:14.527341 5134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711733a7d30d48fc8a924ee54234b980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:14.527747 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.517893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:34355 (local address 127.0.0.1:52402)
0812 11:20:14.518781 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.518794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.518869 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.519578 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.519581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.519600 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:14.519682 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.519688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.521330 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.521333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.522261 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.522270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.527383 (+ 5113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.527387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.527467 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.527489 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.527548 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.527612 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":83,"threads_started":1}
I20260812 11:20:14.527859 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.518358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.30.255.129:34355)
0812 11:20:14.519098 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.519109 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.519131 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:20:14.519204 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.519209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.519421 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.519570 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:14.519790 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.521000 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.522401 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.523331 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.527398 (+ 4067us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.527522 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.527595 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.527732 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":115,"threads_started":1}
I20260812 11:20:14.530084 4885 tablet_service.cc:1467] Processing CreateTablet for tablet 711733a7d30d48fc8a924ee54234b980 (DEFAULT_TABLE table=test-table [id=f5c9f53c9d9745ff91b24fce8c938e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:14.530869 4885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711733a7d30d48fc8a924ee54234b980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:14.534154 5535 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Bootstrap starting.
I20260812 11:20:14.535403 5537 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Bootstrap starting.
I20260812 11:20:14.536446 5535 tablet_bootstrap.cc:654] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:14.536862 5538 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Bootstrap starting.
I20260812 11:20:14.537190 5537 tablet_bootstrap.cc:654] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:14.537799 5535 log.cc:824] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:14.538486 5537 log.cc:824] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:14.539014 5538 tablet_bootstrap.cc:654] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:14.540050 5535 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: No bootstrap required, opened a new log
I20260812 11:20:14.540231 5535 ts_tablet_manager.cc:1397] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:14.540395 5538 log.cc:824] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:14.540556 5537 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: No bootstrap required, opened a new log
I20260812 11:20:14.540736 5537 ts_tablet_manager.cc:1397] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260812 11:20:14.543020 5535 raft_consensus.cc:348] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.543236 5535 raft_consensus.cc:374] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:14.543309 5535 raft_consensus.cc:729] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ee408968d945a891fffb8169186c55, State: Initialized, Role: FOLLOWER
I20260812 11:20:14.543431 5537 raft_consensus.cc:348] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.543685 5537 raft_consensus.cc:374] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:14.543748 5537 raft_consensus.cc:729] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ab7537b29640508b6691d7ab9ad2c2, State: Initialized, Role: FOLLOWER
I20260812 11:20:14.543707 5535 consensus_queue.cc:260] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.544214 5537 consensus_queue.cc:260] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.544381 5511 heartbeater.cc:503] Master 127.30.255.190:44773 was elected leader, sending a full tablet report...
I20260812 11:20:14.544701 5535 ts_tablet_manager.cc:1428] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 11:20:14.545104 5538 tablet_bootstrap.cc:492] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: No bootstrap required, opened a new log
I20260812 11:20:14.545176 5537 ts_tablet_manager.cc:1428] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 11:20:14.545295 5538 ts_tablet_manager.cc:1397] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260812 11:20:14.545475 5246 heartbeater.cc:503] Master 127.30.255.190:44773 was elected leader, sending a full tablet report...
I20260812 11:20:14.548373 5538 raft_consensus.cc:348] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.548584 5538 raft_consensus.cc:374] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:14.548666 5538 raft_consensus.cc:729] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff5cd9a436f4499b2d9a955c58b9e20, State: Initialized, Role: FOLLOWER
I20260812 11:20:14.549081 5538 consensus_queue.cc:260] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.549681 5002 heartbeater.cc:503] Master 127.30.255.190:44773 was elected leader, sending a full tablet report...
I20260812 11:20:14.549933 5538 ts_tablet_manager.cc:1428] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:20:14.557075 5543 raft_consensus.cc:482] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:14.557190 5543 raft_consensus.cc:504] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.558007 5543 leader_election.cc:288] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72ab7537b29640508b6691d7ab9ad2c2 (127.30.255.130:37425), eff5cd9a436f4499b2d9a955c58b9e20 (127.30.255.129:34355)
I20260812 11:20:14.561148 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.558200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:34355 (local address 127.30.255.131:51951)
0812 11:20:14.558305 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.558321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.558401 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.558605 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.558608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.558624 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:14.558726 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.558731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.559433 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.559436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.560195 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.560203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.560778 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.560781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.560882 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.560899 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.560964 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.561026 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 11:20:14.561151 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.558268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:51951 (local address 127.30.255.129:34355)
0812 11:20:14.558364 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.558369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.558381 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:14.558490 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.558494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.558529 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.558589 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:14.558818 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.559341 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.560308 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.560679 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.560778 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.560826 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.560987 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.561059 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:20:14.561581 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.558199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:37425 (local address 127.30.255.131:60849)
0812 11:20:14.558605 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.558616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:14.558712 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:14.558918 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:14.558921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:14.558937 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:14.559045 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.559050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.559763 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.559767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.560517 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:14.560526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.561145 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.561149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:14.561243 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.561266 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:20:14.561377 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 11:20:14.561445 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":98,"thread_start_us":114,"threads_started":1}
I20260812 11:20:14.561584 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.558334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:60849 (local address 127.30.255.130:37425)
0812 11:20:14.558440 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.558445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.558693 (+ 248us) server_negotiation.cc:407] Connection header received
0812 11:20:14.558799 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.558802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.558841 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.558900 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:14.559141 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.559662 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.560647 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.561033 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.561134 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.561179 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.561404 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.561476 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:20:14.561760 4922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711733a7d30d48fc8a924ee54234b980" candidate_uuid: "12ee408968d945a891fffb8169186c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" is_pre_election: true
I20260812 11:20:14.562006 4922 raft_consensus.cc:2454] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ee408968d945a891fffb8169186c55 in term 0.
I20260812 11:20:14.562111 5159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711733a7d30d48fc8a924ee54234b980" candidate_uuid: "12ee408968d945a891fffb8169186c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ab7537b29640508b6691d7ab9ad2c2" is_pre_election: true
I20260812 11:20:14.562326 5159 raft_consensus.cc:2454] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ee408968d945a891fffb8169186c55 in term 0.
I20260812 11:20:14.562436 5294 leader_election.cc:302] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ee408968d945a891fffb8169186c55, eff5cd9a436f4499b2d9a955c58b9e20; no voters:
I20260812 11:20:14.562671 5543 raft_consensus.cc:2781] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:14.562741 5543 raft_consensus.cc:482] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:14.562793 5543 raft_consensus.cc:3037] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:14.563882 5543 raft_consensus.cc:504] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.564172 5543 leader_election.cc:288] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [CANDIDATE]: Term 1 election: Requested vote from peers 72ab7537b29640508b6691d7ab9ad2c2 (127.30.255.130:37425), eff5cd9a436f4499b2d9a955c58b9e20 (127.30.255.129:34355)
I20260812 11:20:14.564455 5159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711733a7d30d48fc8a924ee54234b980" candidate_uuid: "12ee408968d945a891fffb8169186c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ab7537b29640508b6691d7ab9ad2c2"
I20260812 11:20:14.564558 5159 raft_consensus.cc:3037] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:14.564514 4922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711733a7d30d48fc8a924ee54234b980" candidate_uuid: "12ee408968d945a891fffb8169186c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff5cd9a436f4499b2d9a955c58b9e20"
I20260812 11:20:14.564725 4922 raft_consensus.cc:3037] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:14.565501 5159 raft_consensus.cc:2454] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ee408968d945a891fffb8169186c55 in term 1.
I20260812 11:20:14.565649 4922 raft_consensus.cc:2454] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ee408968d945a891fffb8169186c55 in term 1.
I20260812 11:20:14.565811 5297 leader_election.cc:302] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ee408968d945a891fffb8169186c55, 72ab7537b29640508b6691d7ab9ad2c2; no voters:
I20260812 11:20:14.565968 5543 raft_consensus.cc:2781] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:14.566146 5543 raft_consensus.cc:686] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 1 LEADER]: Becoming Leader. State: Replica: 12ee408968d945a891fffb8169186c55, State: Running, Role: LEADER
I20260812 11:20:14.566417 5543 consensus_queue.cc:237] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } }
I20260812 11:20:14.569151 4619 catalog_manager.cc:5495] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ee408968d945a891fffb8169186c55 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "12ee408968d945a891fffb8169186c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ee408968d945a891fffb8169186c55" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 } health_report { overall_health: UNKNOWN } } }
W20260812 11:20:14.578148 5003 tablet.cc:2369] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:14.582774 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.579956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.30.255.131:40903)
0812 11:20:14.580071 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.580074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.580087 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:14.580220 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.580222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.580266 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.580343 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:14.580522 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.580965 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.581707 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.581906 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.581969 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.582173 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:14.582508 (+ 335us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:14.582595 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.582625 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.582666 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:20:14.587639 4922 raft_consensus.cc:1261] T 711733a7d30d48fc8a924ee54234b980 P eff5cd9a436f4499b2d9a955c58b9e20 [term 1 FOLLOWER]: Refusing update from remote peer 12ee408968d945a891fffb8169186c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:14.587651 5159 raft_consensus.cc:1261] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 1 FOLLOWER]: Refusing update from remote peer 12ee408968d945a891fffb8169186c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:14.588238 5554 consensus_queue.cc:1035] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eff5cd9a436f4499b2d9a955c58b9e20" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 34355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:14.588392 5543 consensus_queue.cc:1035] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ab7537b29640508b6691d7ab9ad2c2" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:14.597168 5566 mvcc.cc:205] Tried to move back new op lower bound from 7317641685345484800 to 7317641685266489344. Current Snapshot: MvccSnapshot[applied={T|T < 7317641685345484800}]
I20260812 11:20:14.604368 4613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:14.604501 4613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:14.606695 4613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133912cac07b4e16b9c298011b7285b7: Sending DeleteTablet for 3 replicas of tablet 711733a7d30d48fc8a924ee54234b980
I20260812 11:20:14.607368 5134 tablet_service.cc:1514] Processing DeleteTablet for tablet 711733a7d30d48fc8a924ee54234b980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:14 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:47218
I20260812 11:20:14.607548 4885 tablet_service.cc:1514] Processing DeleteTablet for tablet 711733a7d30d48fc8a924ee54234b980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:14 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:52402
I20260812 11:20:14.608059 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4744
I20260812 11:20:14.608141 5378 tablet_service.cc:1514] Processing DeleteTablet for tablet 711733a7d30d48fc8a924ee54234b980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:14 UTC) from {username='kudu', principal='kudu/127.30.255.190@KRBTEST.COM'} at 127.0.0.1:59178
I20260812 11:20:14.622682 5580 tablet_replica.cc:330] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: stopping tablet replica
I20260812 11:20:14.623028 5580 raft_consensus.cc:2229] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:14.623375 5580 raft_consensus.cc:2258] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:14.623790 5584 tablet_replica.cc:330] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: stopping tablet replica
I20260812 11:20:14.624125 5584 raft_consensus.cc:2229] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:20:14.624540 5584 raft_consensus.cc:2258] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:14.624951 5580 ts_tablet_manager.cc:1905] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:14.625957 5584 ts_tablet_manager.cc:1905] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:14.627730 5580 ts_tablet_manager.cc:1918] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:14.627892 5580 log.cc:1196] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/711733a7d30d48fc8a924ee54234b980
I20260812 11:20:14.628305 5580 ts_tablet_manager.cc:1939] T 711733a7d30d48fc8a924ee54234b980 P 72ab7537b29640508b6691d7ab9ad2c2: Deleting consensus metadata
I20260812 11:20:14.628734 5584 ts_tablet_manager.cc:1918] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:14.628918 5584 log.cc:1196] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/711733a7d30d48fc8a924ee54234b980
I20260812 11:20:14.629284 5584 ts_tablet_manager.cc:1939] T 711733a7d30d48fc8a924ee54234b980 P 12ee408968d945a891fffb8169186c55: Deleting consensus metadata
I20260812 11:20:14.629840 4590 catalog_manager.cc:4841] TS 72ab7537b29640508b6691d7ab9ad2c2 (127.30.255.130:37425): tablet 711733a7d30d48fc8a924ee54234b980 (table test-table [id=f5c9f53c9d9745ff91b24fce8c938e2a]) successfully deleted
I20260812 11:20:14.630373 4581 catalog_manager.cc:4841] TS 12ee408968d945a891fffb8169186c55 (127.30.255.131:40903): tablet 711733a7d30d48fc8a924ee54234b980 (table test-table [id=f5c9f53c9d9745ff91b24fce8c938e2a]) successfully deleted
W20260812 11:20:14.631472 4581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:34355 (error 108)
I20260812 11:20:14.631556 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5010
I20260812 11:20:14.637177 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.636882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:34355 (local address 127.0.0.1:52414)
0812 11:20:14.637012 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:14.637089 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:34355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 11:20:14.637336 4581 catalog_manager.cc:4584] TS eff5cd9a436f4499b2d9a955c58b9e20 (127.30.255.129:34355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 711733a7d30d48fc8a924ee54234b980: Network error: Client connection negotiation failed: client connection to 127.30.255.129:34355: connect: Connection refused (error 111)
I20260812 11:20:14.641717 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5257
I20260812 11:20:14.658306 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 4548
2026-08-12T11:20:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2357 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T11:20:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:14Z Disabled control of system clock
I20260812 11:20:14.696372 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37045
--webserver_interface=127.30.255.190
--webserver_port=0
--builtin_ntp_servers=127.30.255.148:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:37045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:14.802693 5595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:14.802917 5595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:14.802949 5595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:14.806829 5595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:14.806879 5595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:14.806895 5595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:14.806922 5595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:14.806950 5595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:14.811563 5595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:37045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:37045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:14.812461 5595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:14.813723 5595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:14.821527 5606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.821584 5607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:14.821678 5609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:14.821761 5595 server_base.cc:1034] running on GCE node
I20260812 11:20:14.822585 5595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:14.823635 5595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:14.824790 5595 hybrid_clock.cc:648] HybridClock initialized: now 1786533614824776 us; error 27 us; skew 500 ppm
I20260812 11:20:14.826470 5595 webserver.cc:460] Webserver started at http://127.30.255.190:33773/ using document root <none> and password file <none>
I20260812 11:20:14.827083 5595 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:14.827147 5595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:14.827404 5595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:14.829423 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "8e408f471c324ff794140bc4f17abaa6"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "ca627425448dbba3508a3675703d0742"
server_key_iv: "2c015a7a32e5659d673c6a050b02a7fa"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.829965 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "8e408f471c324ff794140bc4f17abaa6"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "ca627425448dbba3508a3675703d0742"
server_key_iv: "2c015a7a32e5659d673c6a050b02a7fa"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.833511 5595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260812 11:20:14.835757 5620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:14.836869 5595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:14.836978 5595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "8e408f471c324ff794140bc4f17abaa6"
format_stamp: "Formatted at 2026-08-12 11:20:14 on dist-test-slave-gjrz"
server_key: "ca627425448dbba3508a3675703d0742"
server_key_iv: "2c015a7a32e5659d673c6a050b02a7fa"
server_key_version: "encryptionkey@0"
I20260812 11:20:14.837064 5595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:14.852253 5595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:14.852869 5595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:14.853030 5595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:14.859462 5595 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:37045
I20260812 11:20:14.859493 5720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:37045 every 8 connection(s)
I20260812 11:20:14.860337 5595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:14.861375 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5595
I20260812 11:20:14.861495 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:14.861763 31742 external_mini_cluster.cc:1428] Setting key e0485e0f6ea791897aa01c5f5a172d68
I20260812 11:20:14.863732 5722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:14.870499 5722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: Bootstrap starting.
I20260812 11:20:14.871821 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:14.863112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.30.255.190:37045)
0812 11:20:14.863805 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:14.863830 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:14.863874 (+ 44us) server_negotiation.cc:407] Connection header received
0812 11:20:14.864714 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:14.864736 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:14.865215 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:14.865571 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:14.866002 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.867476 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.868316 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:14.868996 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:14.869159 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:14.869861 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:14.869868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:14.869883 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:14.870351 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:14.870420 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:14.870445 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:14.870545 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:14.870636 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 11:20:14.870895 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":152,"threads_started":1}
I20260812 11:20:14.872964 5722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:14.873819 5722 log.cc:824] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:14.875634 5722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: No bootstrap required, opened a new log
I20260812 11:20:14.879066 5722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } }
I20260812 11:20:14.879297 5722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:14.879354 5722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e408f471c324ff794140bc4f17abaa6, State: Initialized, Role: FOLLOWER
I20260812 11:20:14.879699 5722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } }
I20260812 11:20:14.879810 5722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:14.879853 5722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:14.879920 5722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:14.880664 5722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } }
I20260812 11:20:14.880939 5722 leader_election.cc:302] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e408f471c324ff794140bc4f17abaa6; no voters:
I20260812 11:20:14.881191 5722 leader_election.cc:288] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:14.881287 5731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:14.881485 5731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [term 1 LEADER]: Becoming Leader. State: Replica: 8e408f471c324ff794140bc4f17abaa6, State: Running, Role: LEADER
I20260812 11:20:14.881810 5731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } }
I20260812 11:20:14.882274 5722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:14.883240 5732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e408f471c324ff794140bc4f17abaa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } } }
I20260812 11:20:14.883394 5732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:14.883364 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e408f471c324ff794140bc4f17abaa6. Latest consensus state: current_term: 1 leader_uuid: "8e408f471c324ff794140bc4f17abaa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e408f471c324ff794140bc4f17abaa6" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 37045 } } }
I20260812 11:20:14.883462 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:14.883862 5741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:14.886726 5741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:14.891657 5741 catalog_manager.cc:1347] Generated new cluster ID: 9b952e044c7546d4bb5e7425a49de73c
I20260812 11:20:14.891757 5741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:14.908155 5741 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:14.909060 5741 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:14.914202 5741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: Generated new TSK 0
I20260812 11:20:14.914762 5741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:14.919749 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.129:0
--local_ip_for_outbound_sockets=127.30.255.129
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--builtin_ntp_servers=127.30.255.148:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:20:15.029939 5761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:15.030153 5761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:15.030186 5761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:15.034034 5761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:15.034080 5761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:15.034128 5761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.129
I20260812 11:20:15.038602 5761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.255.129
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.129
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:15.039525 5761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:15.040779 5761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:15.049875 5778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:15.049909 5761 server_base.cc:1034] running on GCE node
W20260812 11:20:15.049851 5776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:15.049857 5772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:15.050633 5761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:15.051175 5761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:15.052337 5761 hybrid_clock.cc:648] HybridClock initialized: now 1786533615052314 us; error 27 us; skew 500 ppm
I20260812 11:20:15.054014 5761 webserver.cc:460] Webserver started at http://127.30.255.129:44049/ using document root <none> and password file <none>
I20260812 11:20:15.054526 5761 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:15.054584 5761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:15.054776 5761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:15.056790 5761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data/instance:
uuid: "670e1757db664b69bf91f4e538a420c1"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "1535361d61a7b3bccfaea91e4572e489"
server_key_iv: "5a9317efba503a66a625ad43f5343dee"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.057296 5761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance:
uuid: "670e1757db664b69bf91f4e538a420c1"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "1535361d61a7b3bccfaea91e4572e489"
server_key_iv: "5a9317efba503a66a625ad43f5343dee"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.060688 5761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:15.062870 5789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.063990 5761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:15.064091 5761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal
uuid: "670e1757db664b69bf91f4e538a420c1"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "1535361d61a7b3bccfaea91e4572e489"
server_key_iv: "5a9317efba503a66a625ad43f5343dee"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.064177 5761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:15.082680 5761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:15.083252 5761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:15.083461 5761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:15.083997 5761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:15.084884 5761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:15.084944 5761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.084995 5761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:15.085031 5761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.093073 5761 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.129:32877
I20260812 11:20:15.093112 6029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.129:32877 every 8 connection(s)
I20260812 11:20:15.093843 5761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/data/info.pb
I20260812 11:20:15.096439 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 5761
I20260812 11:20:15.096575 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-0/wal/instance
I20260812 11:20:15.096843 31742 external_mini_cluster.cc:1428] Setting key 3f1f1c374b8d9996e58483346f58cea3
I20260812 11:20:15.098881 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.130:0
--local_ip_for_outbound_sockets=127.30.255.130
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--builtin_ntp_servers=127.30.255.148:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:15.103121 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.095489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.129:55429 (local address 127.30.255.190:37045)
0812 11:20:15.095609 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.095613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.096573 (+ 960us) server_negotiation.cc:407] Connection header received
0812 11:20:15.098114 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.098118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.098176 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.098266 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:15.099370 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.099899 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.101259 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.101589 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.101696 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.102648 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:15.102651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:15.102653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:15.102665 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:15.102685 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:15.102693 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:15.102753 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.102891 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.102976 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:15.103935 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.095834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37045 (local address 127.30.255.129:55429)
0812 11:20:15.096437 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.096475 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.097879 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.098459 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.098468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.098569 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:15.099216 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.099236 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.100034 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.100038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.101115 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.101135 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.101721 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.101724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.101859 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.102398 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:15.102399 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:15.102435 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:15.102495 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:15.102764 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:15.102773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:15.102779 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.102881 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.103156 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":107,"threads_started":1}
I20260812 11:20:15.105039 6030 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37045
I20260812 11:20:15.105283 6030 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:15.105868 6030 heartbeater.cc:511] Master 127.30.255.190:37045 requested a full tablet report, sending...
I20260812 11:20:15.107331 5650 ts_manager.cc:194] Registered new tserver with Master: 670e1757db664b69bf91f4e538a420c1 (127.30.255.129:32877)
I20260812 11:20:15.108443 5650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.129:55429
W20260812 11:20:15.209385 6045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:15.209704 6045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:15.209760 6045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:15.215660 6045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:15.215736 6045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:15.215823 6045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.130
I20260812 11:20:15.221650 6045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.255.130
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.130
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:15.222702 6045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:15.224092 6045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:15.233230 6054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:15.233276 6050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:15.233392 6045 server_base.cc:1034] running on GCE node
W20260812 11:20:15.233237 6063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:15.233886 6045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:15.234515 6045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:15.235671 6045 hybrid_clock.cc:648] HybridClock initialized: now 1786533615235646 us; error 38 us; skew 500 ppm
I20260812 11:20:15.237370 6045 webserver.cc:460] Webserver started at http://127.30.255.130:33009/ using document root <none> and password file <none>
I20260812 11:20:15.237921 6045 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:15.237989 6045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:15.238197 6045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:15.240336 6045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data/instance:
uuid: "78dcdd8a42b642a780dca1e7fc476a11"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ab7360b7b7ad92bb0012211ec5fd8045"
server_key_iv: "79d88b9dff68996e701e07cc7273dc74"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.240876 6045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance:
uuid: "78dcdd8a42b642a780dca1e7fc476a11"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ab7360b7b7ad92bb0012211ec5fd8045"
server_key_iv: "79d88b9dff68996e701e07cc7273dc74"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.244453 6045 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:15.246757 6079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.247905 6045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:15.248003 6045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal
uuid: "78dcdd8a42b642a780dca1e7fc476a11"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ab7360b7b7ad92bb0012211ec5fd8045"
server_key_iv: "79d88b9dff68996e701e07cc7273dc74"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.248097 6045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:15.266039 6045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:15.266665 6045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:15.266829 6045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:15.267408 6045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:15.268361 6045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:15.268409 6045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.268450 6045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:15.268477 6045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.276567 6045 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.130:42239
I20260812 11:20:15.276636 6295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.130:42239 every 8 connection(s)
I20260812 11:20:15.277408 6045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/data/info.pb
I20260812 11:20:15.278139 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6045
I20260812 11:20:15.278273 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal/instance
I20260812 11:20:15.278561 31742 external_mini_cluster.cc:1428] Setting key 81594a9d9d87b8912a380b34efd7aa6f
I20260812 11:20:15.280607 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.255.131:0
--local_ip_for_outbound_sockets=127.30.255.131
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--builtin_ntp_servers=127.30.255.148:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:20:15.291457 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.279357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.130:52205 (local address 127.30.255.190:37045)
0812 11:20:15.279929 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.279934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.280681 (+ 747us) server_negotiation.cc:407] Connection header received
0812 11:20:15.282104 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.282108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.282166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.282255 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:15.283467 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.284093 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.285208 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.285461 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.285552 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.290960 (+ 5408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:15.290963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:15.290965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:15.290977 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:15.290995 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:15.291003 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:15.291071 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.291217 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.291308 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
I20260812 11:20:15.292311 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.279755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37045 (local address 127.30.255.130:52205)
0812 11:20:15.280510 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.280560 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.281872 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.282450 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.282459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.282598 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:15.283310 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.283331 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.284211 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.284214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.285090 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.285101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.285613 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.285618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.285783 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.290711 (+ 4928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:15.290713 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:15.290752 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:15.290816 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:15.291074 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:15.291083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:15.291089 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.291192 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.291497 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":152,"threads_started":1}
I20260812 11:20:15.293453 6300 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37045
I20260812 11:20:15.293665 6300 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:15.294134 6300 heartbeater.cc:511] Master 127.30.255.190:37045 requested a full tablet report, sending...
I20260812 11:20:15.295109 5650 ts_manager.cc:194] Registered new tserver with Master: 78dcdd8a42b642a780dca1e7fc476a11 (127.30.255.130:42239)
I20260812 11:20:15.295882 5650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.130:52205
W20260812 11:20:15.391510 6307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:15.391737 6307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:15.391772 6307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:15.395602 6307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:15.395651 6307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:15.395699 6307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.255.131
I20260812 11:20:15.400694 6307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.255.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.255.131
--webserver_port=0
--tserver_master_addrs=127.30.255.190:37045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.255.131
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:15.401578 6307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:15.402822 6307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:20:15.411685 6307 server_base.cc:1034] running on GCE node
W20260812 11:20:15.411650 6317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:15.411666 6319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:15.411629 6321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:15.412248 6307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:15.412791 6307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:15.413956 6307 hybrid_clock.cc:648] HybridClock initialized: now 1786533615413932 us; error 37 us; skew 500 ppm
I20260812 11:20:15.415632 6307 webserver.cc:460] Webserver started at http://127.30.255.131:36253/ using document root <none> and password file <none>
I20260812 11:20:15.416126 6307 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:15.416190 6307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:15.416363 6307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:15.418226 6307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data/instance:
uuid: "ddaba959c3ff49cc9521261356b51d14"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ae189c5c3bb602ebd2c3f901dd68b247"
server_key_iv: "91e8cd449554beedf371ee8ad20f2cfd"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.418735 6307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance:
uuid: "ddaba959c3ff49cc9521261356b51d14"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ae189c5c3bb602ebd2c3f901dd68b247"
server_key_iv: "91e8cd449554beedf371ee8ad20f2cfd"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.422065 6307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 11:20:15.424258 6339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.425303 6307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:15.425412 6307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal
uuid: "ddaba959c3ff49cc9521261356b51d14"
format_stamp: "Formatted at 2026-08-12 11:20:15 on dist-test-slave-gjrz"
server_key: "ae189c5c3bb602ebd2c3f901dd68b247"
server_key_iv: "91e8cd449554beedf371ee8ad20f2cfd"
server_key_version: "encryptionkey@0"
I20260812 11:20:15.425496 6307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:15.440608 6307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:15.441154 6307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:15.441303 6307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:15.441838 6307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:20:15.442691 6307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:20:15.442739 6307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.442788 6307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:20:15.442816 6307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:15.450987 6307 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.131:40193
I20260812 11:20:15.451016 6564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.131:40193 every 8 connection(s)
I20260812 11:20:15.451825 6307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/data/info.pb
I20260812 11:20:15.457300 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6307
I20260812 11:20:15.457404 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal/instance
I20260812 11:20:15.457665 31742 external_mini_cluster.cc:1428] Setting key 8432b676119c28c1f8e9d32bf742986d
I20260812 11:20:15.460631 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.453476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:50029 (local address 127.30.255.190:37045)
0812 11:20:15.453605 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.453609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.454598 (+ 989us) server_negotiation.cc:407] Connection header received
0812 11:20:15.455971 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.455975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.456034 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.456109 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:15.457032 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.457626 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.458787 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.459005 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.459096 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.460180 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:15.460183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:15.460185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:15.460197 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:15.460214 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:15.460222 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:15.460280 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.460428 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.460490 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:20:15.461418 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.453824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:37045 (local address 127.30.255.131:50029)
0812 11:20:15.454456 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.454495 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.455753 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.456287 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.456293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.456375 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:15.456883 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.456897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.457759 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.457762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.458659 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.458669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.459145 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.459149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.459358 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.459926 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:20:15.459928 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:15.459966 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:20:15.460028 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:15.460292 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:15.460302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:15.460308 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.460417 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.460699 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
I20260812 11:20:15.462405 6565 heartbeater.cc:347] Connected to a master server at 127.30.255.190:37045
I20260812 11:20:15.462596 6565 heartbeater.cc:464] Registering TS with master...
I20260812 11:20:15.463045 6565 heartbeater.cc:511] Master 127.30.255.190:37045 requested a full tablet report, sending...
I20260812 11:20:15.463967 5650 ts_manager.cc:194] Registered new tserver with Master: ddaba959c3ff49cc9521261356b51d14 (127.30.255.131:40193)
I20260812 11:20:15.464489 5650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.255.131:50029
I20260812 11:20:15.470955 31742 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:20:15.475845 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.472786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.30.255.190:37045)
0812 11:20:15.472883 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.472886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.473067 (+ 181us) server_negotiation.cc:407] Connection header received
0812 11:20:15.473158 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.473160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.473196 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.473253 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:15.473443 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.473910 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.474642 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.474834 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.474896 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.475572 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:15.475575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:15.475577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:15.475587 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:15.475601 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:15.475608 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:15.475658 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.475708 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.475748 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:20:15.478446 5650 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:20:15.481396 5650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:20:15.495635 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.489104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:32877 (local address 127.0.0.1:49878)
0812 11:20:15.489813 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.489860 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.489994 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.490784 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.490790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.490813 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:15.490943 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.490951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492615 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.492619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.494097 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.494110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.495101 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.495105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.495218 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.495284 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.495370 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.495483 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":114,"thread_start_us":71,"threads_started":1}
I20260812 11:20:15.495945 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.490242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.30.255.131:40193)
0812 11:20:15.490574 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.490585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.490879 (+ 294us) server_negotiation.cc:407] Connection header received
0812 11:20:15.490950 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.490954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.491151 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.491543 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:15.491683 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492714 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.494206 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.494969 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.495080 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.495247 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.495704 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.495795 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":145,"threads_started":1}
I20260812 11:20:15.496156 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.489861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.131:40193 (local address 127.0.0.1:42268)
0812 11:20:15.490689 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.490701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.490794 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.491320 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.491323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.491341 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:15.491422 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.491429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492831 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.492834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.494062 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.494074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.495328 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.495332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.495420 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.495444 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.495509 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.495574 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":5,"thread_start_us":88,"threads_started":1}
I20260812 11:20:15.496462 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.489224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.30.255.129:32877)
0812 11:20:15.489861 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.489873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.490295 (+ 422us) server_negotiation.cc:407] Connection header received
0812 11:20:15.490379 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.490383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.490590 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.491099 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:15.491300 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492464 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.494237 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.494966 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.496006 (+ 1040us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.496168 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.496218 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.496306 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":100,"threads_started":1}
I20260812 11:20:15.497779 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.489250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:42239 (local address 127.0.0.1:51200)
0812 11:20:15.490225 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.490237 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.490325 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.490989 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.490992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.491008 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:15.491100 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.491107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492577 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.492580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.495934 (+ 3354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.495946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.497162 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.497166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.497253 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.497274 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.497343 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.497404 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":980,"thread_start_us":76,"threads_started":1}
I20260812 11:20:15.498306 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.489417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51200 (local address 127.30.255.130:42239)
0812 11:20:15.490115 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.490129 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.490428 (+ 299us) server_negotiation.cc:407] Connection header received
0812 11:20:15.490513 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.490519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.490705 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.490849 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:15.491250 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.492450 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.496077 (+ 3627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.496817 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.496933 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.497085 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.498049 (+ 964us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.498153 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":125,"threads_started":1}
I20260812 11:20:15.499214 6433 tablet_service.cc:1467] Processing CreateTablet for tablet 9d858c3c239d4439b12bc8e52d926332 (DEFAULT_TABLE table=test-table [id=5df9ce28badf4309b5b8ca104e59ee1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:15.499213 5891 tablet_service.cc:1467] Processing CreateTablet for tablet 9d858c3c239d4439b12bc8e52d926332 (DEFAULT_TABLE table=test-table [id=5df9ce28badf4309b5b8ca104e59ee1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:15.500203 6433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d858c3c239d4439b12bc8e52d926332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:15.500248 5891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d858c3c239d4439b12bc8e52d926332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:15.501048 6172 tablet_service.cc:1467] Processing CreateTablet for tablet 9d858c3c239d4439b12bc8e52d926332 (DEFAULT_TABLE table=test-table [id=5df9ce28badf4309b5b8ca104e59ee1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:20:15.501983 6172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d858c3c239d4439b12bc8e52d926332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:15.506832 6600 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Bootstrap starting.
I20260812 11:20:15.507411 6601 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Bootstrap starting.
I20260812 11:20:15.508256 6604 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Bootstrap starting.
I20260812 11:20:15.509292 6600 tablet_bootstrap.cc:654] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:15.509311 6601 tablet_bootstrap.cc:654] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:15.509968 6604 tablet_bootstrap.cc:654] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:15.510159 6601 log.cc:824] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:15.510159 6600 log.cc:824] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:15.510695 6604 log.cc:824] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:15.512197 6601 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: No bootstrap required, opened a new log
I20260812 11:20:15.512197 6600 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: No bootstrap required, opened a new log
I20260812 11:20:15.512389 6601 ts_tablet_manager.cc:1397] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 11:20:15.512400 6600 ts_tablet_manager.cc:1397] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:15.512914 6604 tablet_bootstrap.cc:492] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: No bootstrap required, opened a new log
I20260812 11:20:15.513087 6604 ts_tablet_manager.cc:1397] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 11:20:15.515736 6604 raft_consensus.cc:348] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.515930 6604 raft_consensus.cc:374] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:15.515981 6604 raft_consensus.cc:729] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dcdd8a42b642a780dca1e7fc476a11, State: Initialized, Role: FOLLOWER
I20260812 11:20:15.516364 6604 consensus_queue.cc:260] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.516469 6600 raft_consensus.cc:348] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.516469 6601 raft_consensus.cc:348] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.516716 6600 raft_consensus.cc:374] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:15.516716 6601 raft_consensus.cc:374] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:15.516774 6601 raft_consensus.cc:729] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670e1757db664b69bf91f4e538a420c1, State: Initialized, Role: FOLLOWER
I20260812 11:20:15.516783 6600 raft_consensus.cc:729] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddaba959c3ff49cc9521261356b51d14, State: Initialized, Role: FOLLOWER
I20260812 11:20:15.517077 6300 heartbeater.cc:503] Master 127.30.255.190:37045 was elected leader, sending a full tablet report...
I20260812 11:20:15.517278 6604 ts_tablet_manager.cc:1428] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260812 11:20:15.517217 6601 consensus_queue.cc:260] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.517204 6600 consensus_queue.cc:260] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.518124 6601 ts_tablet_manager.cc:1428] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:20:15.518139 6600 ts_tablet_manager.cc:1428] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:20:15.518455 6565 heartbeater.cc:503] Master 127.30.255.190:37045 was elected leader, sending a full tablet report...
I20260812 11:20:15.519030 6030 heartbeater.cc:503] Master 127.30.255.190:37045 was elected leader, sending a full tablet report...
W20260812 11:20:15.528245 6304 tablet.cc:2369] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:20:15.594882 6033 tablet.cc:2369] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:20:15.702741 6567 tablet.cc:2369] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:15.739094 6613 raft_consensus.cc:482] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:20:15.739341 6613 raft_consensus.cc:504] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.740517 6613 leader_election.cc:288] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 670e1757db664b69bf91f4e538a420c1 (127.30.255.129:32877), 78dcdd8a42b642a780dca1e7fc476a11 (127.30.255.130:42239)
I20260812 11:20:15.744469 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.740890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:45361 (local address 127.30.255.129:32877)
0812 11:20:15.741020 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.741025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.741354 (+ 329us) server_negotiation.cc:407] Connection header received
0812 11:20:15.741474 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.741477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.741526 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.741589 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:15.741901 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.742390 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.743492 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.743890 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.743999 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.744052 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.744276 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.744348 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:20:15.744469 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.740779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.130:42239 (local address 127.30.255.131:55335)
0812 11:20:15.741049 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.741066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.741173 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.741532 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.741535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.741562 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:15.741664 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.741669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.742444 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.742448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.743383 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.743393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.744007 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.744011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.744135 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.744180 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.744245 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.744318 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":65}
I20260812 11:20:15.744474 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.740932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.255.131:55335 (local address 127.30.255.130:42239)
0812 11:20:15.741037 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.741041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.741140 (+ 99us) server_negotiation.cc:407] Connection header received
0812 11:20:15.741289 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.741292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.741345 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.741414 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:20:15.741783 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.742330 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.743514 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.743892 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.743999 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.744052 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.744282 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.744355 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:15.744488 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.740779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:32877 (local address 127.30.255.131:45361)
0812 11:20:15.741240 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.741255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:15.741382 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:15.741615 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:15.741624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:15.741641 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:20:15.741767 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.741775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.742505 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.742507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.743371 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:15.743381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.744007 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.744010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:15.744150 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.744181 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 11:20:15.744250 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 11:20:15.744320 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":88,"thread_start_us":167,"threads_started":1}
I20260812 11:20:15.745157 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d858c3c239d4439b12bc8e52d926332" candidate_uuid: "ddaba959c3ff49cc9521261356b51d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dcdd8a42b642a780dca1e7fc476a11" is_pre_election: true
I20260812 11:20:15.745407 5944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d858c3c239d4439b12bc8e52d926332" candidate_uuid: "ddaba959c3ff49cc9521261356b51d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670e1757db664b69bf91f4e538a420c1" is_pre_election: true
I20260812 11:20:15.745481 6203 raft_consensus.cc:2454] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddaba959c3ff49cc9521261356b51d14 in term 0.
I20260812 11:20:15.745688 5944 raft_consensus.cc:2454] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddaba959c3ff49cc9521261356b51d14 in term 0.
I20260812 11:20:15.746019 6344 leader_election.cc:302] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78dcdd8a42b642a780dca1e7fc476a11, ddaba959c3ff49cc9521261356b51d14; no voters:
I20260812 11:20:15.746289 6613 raft_consensus.cc:2781] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:20:15.746376 6613 raft_consensus.cc:482] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:20:15.746424 6613 raft_consensus.cc:3037] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:15.747433 6613 raft_consensus.cc:504] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.747743 6613 leader_election.cc:288] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [CANDIDATE]: Term 1 election: Requested vote from peers 670e1757db664b69bf91f4e538a420c1 (127.30.255.129:32877), 78dcdd8a42b642a780dca1e7fc476a11 (127.30.255.130:42239)
I20260812 11:20:15.748032 5944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d858c3c239d4439b12bc8e52d926332" candidate_uuid: "ddaba959c3ff49cc9521261356b51d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670e1757db664b69bf91f4e538a420c1"
I20260812 11:20:15.748083 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d858c3c239d4439b12bc8e52d926332" candidate_uuid: "ddaba959c3ff49cc9521261356b51d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dcdd8a42b642a780dca1e7fc476a11"
I20260812 11:20:15.748153 5944 raft_consensus.cc:3037] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:15.748178 6203 raft_consensus.cc:3037] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:15.749352 6203 raft_consensus.cc:2454] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddaba959c3ff49cc9521261356b51d14 in term 1.
I20260812 11:20:15.749352 5944 raft_consensus.cc:2454] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddaba959c3ff49cc9521261356b51d14 in term 1.
I20260812 11:20:15.749637 6344 leader_election.cc:302] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78dcdd8a42b642a780dca1e7fc476a11, ddaba959c3ff49cc9521261356b51d14; no voters:
I20260812 11:20:15.749807 6613 raft_consensus.cc:2781] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:15.750015 6613 raft_consensus.cc:686] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 1 LEADER]: Becoming Leader. State: Replica: ddaba959c3ff49cc9521261356b51d14, State: Running, Role: LEADER
I20260812 11:20:15.750478 6613 consensus_queue.cc:237] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } }
I20260812 11:20:15.753547 5650 catalog_manager.cc:5495] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddaba959c3ff49cc9521261356b51d14 (127.30.255.131). New cstate: current_term: 1 leader_uuid: "ddaba959c3ff49cc9521261356b51d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddaba959c3ff49cc9521261356b51d14" member_type: VOTER last_known_addr { host: "127.30.255.131" port: 40193 } health_report { overall_health: HEALTHY } } }
I20260812 11:20:15.782275 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.779364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.30.255.131:40193)
0812 11:20:15.779466 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:15.779469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:15.779483 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:20:15.779617 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:15.779619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:15.779670 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:15.779747 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:15.779928 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.780393 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.781148 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:15.781338 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:15.781399 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:15.781617 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:15.781996 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:15.782080 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:15.782137 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 11:20:15.782180 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:20:15.787714 5944 raft_consensus.cc:1261] T 9d858c3c239d4439b12bc8e52d926332 P 670e1757db664b69bf91f4e538a420c1 [term 1 FOLLOWER]: Refusing update from remote peer ddaba959c3ff49cc9521261356b51d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:15.787714 6203 raft_consensus.cc:1261] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 1 FOLLOWER]: Refusing update from remote peer ddaba959c3ff49cc9521261356b51d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:15.788422 6616 consensus_queue.cc:1035] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dcdd8a42b642a780dca1e7fc476a11" member_type: VOTER last_known_addr { host: "127.30.255.130" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:15.788595 6613 consensus_queue.cc:1035] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "670e1757db664b69bf91f4e538a420c1" member_type: VOTER last_known_addr { host: "127.30.255.129" port: 32877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:15.797935 6628 mvcc.cc:205] Tried to move back new op lower bound from 7317641690259484672 to 7317641690116329472. Current Snapshot: MvccSnapshot[applied={T|T < 7317641690259484672}]
I20260812 11:20:15.798372 6624 mvcc.cc:205] Tried to move back new op lower bound from 7317641690259484672 to 7317641690116329472. Current Snapshot: MvccSnapshot[applied={T|T < 7317641690259484672}]
I20260812 11:20:15.802953 5648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:15.803097 5648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:20:15.805323 5648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e408f471c324ff794140bc4f17abaa6: Sending DeleteTablet for 3 replicas of tablet 9d858c3c239d4439b12bc8e52d926332
I20260812 11:20:15.805945 5891 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d858c3c239d4439b12bc8e52d926332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:15 UTC) from {username='slave'} at 127.0.0.1:49878
I20260812 11:20:15.806018 6172 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d858c3c239d4439b12bc8e52d926332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:15 UTC) from {username='slave'} at 127.0.0.1:51200
I20260812 11:20:15.806106 6433 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d858c3c239d4439b12bc8e52d926332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:20:15 UTC) from {username='slave'} at 127.0.0.1:42268
I20260812 11:20:15.806456 6642 tablet_replica.cc:330] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: stopping tablet replica
I20260812 11:20:15.806464 6647 tablet_replica.cc:330] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: stopping tablet replica
I20260812 11:20:15.806730 6642 raft_consensus.cc:2229] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:20:15.806730 6647 raft_consensus.cc:2229] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:20:15.807000 6642 raft_consensus.cc:2258] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:15.807016 6647 raft_consensus.cc:2258] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:20:15.807219 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5761
I20260812 11:20:15.814983 6642 ts_tablet_manager.cc:1905] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:15.816366 6647 ts_tablet_manager.cc:1905] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:20:15.817646 6642 ts_tablet_manager.cc:1918] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:15.817726 6642 log.cc:1196] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-1/wal/wals/9d858c3c239d4439b12bc8e52d926332
I20260812 11:20:15.818005 6642 ts_tablet_manager.cc:1939] T 9d858c3c239d4439b12bc8e52d926332 P 78dcdd8a42b642a780dca1e7fc476a11: Deleting consensus metadata
I20260812 11:20:15.819135 5624 catalog_manager.cc:4841] TS 78dcdd8a42b642a780dca1e7fc476a11 (127.30.255.130:42239): tablet 9d858c3c239d4439b12bc8e52d926332 (table test-table [id=5df9ce28badf4309b5b8ca104e59ee1c]) successfully deleted
I20260812 11:20:15.819509 6647 ts_tablet_manager.cc:1918] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:20:15.819614 6647 log.cc:1196] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Deleting WAL directory at /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786533365564394-31742-0/minicluster-data/ts-2/wal/wals/9d858c3c239d4439b12bc8e52d926332
I20260812 11:20:15.819998 6647 ts_tablet_manager.cc:1939] T 9d858c3c239d4439b12bc8e52d926332 P ddaba959c3ff49cc9521261356b51d14: Deleting consensus metadata
I20260812 11:20:15.821045 5623 catalog_manager.cc:4841] TS ddaba959c3ff49cc9521261356b51d14 (127.30.255.131:40193): tablet 9d858c3c239d4439b12bc8e52d926332 (table test-table [id=5df9ce28badf4309b5b8ca104e59ee1c]) successfully deleted
W20260812 11:20:15.831430 5622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.255.129:32877 (error 108)
I20260812 11:20:15.832239 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6045
I20260812 11:20:15.832365 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:15.832109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.129:32877 (local address 127.0.0.1:49888)
0812 11:20:15.832226 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:15.832284 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.129:32877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 11:20:15.843114 5622 catalog_manager.cc:4584] TS 670e1757db664b69bf91f4e538a420c1 (127.30.255.129:32877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d858c3c239d4439b12bc8e52d926332: Network error: Client connection negotiation failed: client connection to 127.30.255.129:32877: connect: Connection refused (error 111)
I20260812 11:20:15.849260 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6307
I20260812 11:20:15.864763 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 5595
2026-08-12T11:20:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1203 ms)
[----------] 6 tests from EncryptionPolicyTest (10660 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:15 dist-test-slave-gjrz krb5kdc[6658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:15 dist-test-slave-gjrz krb5kdc[6658](info): set up 2 sockets
Aug 12 11:20:15 dist-test-slave-gjrz krb5kdc[6658](info): commencing operation
krb5kdc: starting...
W20260812 11:20:17.064028 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:17.175851 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:17.281176 6679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:17.281402 6679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:17.281456 6679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:17.285333 6679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:17.285383 6679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:17.285409 6679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:17.285429 6679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:17.285459 6679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:17.290359 6679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:17.291235 6679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:17.292141 6679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:17.299937 6700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:17.299966 6694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:17.300014 6697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:17.300071 6679 server_base.cc:1034] running on GCE node
I20260812 11:20:17.300664 6679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:17.301645 6679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:17.302804 6679 hybrid_clock.cc:648] HybridClock initialized: now 1786533617302779 us; error 35 us; skew 500 ppm
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:17.305279 6679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:17.306046 6679 webserver.cc:460] Webserver started at http://127.0.0.1:41067/ using document root <none> and password file <none>
I20260812 11:20:17.306522 6679 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:17.306588 6679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:17.306763 6679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:17.308697 6679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "145aa71aa23f4e60b739ff9460e4218a"
format_stamp: "Formatted at 2026-08-12 11:20:17 on dist-test-slave-gjrz"
server_key: "aa1bf2c44b846a13a3104e5d2fcee6dd"
server_key_iv: "291f7aed13a8888127bd892261bd483c"
server_key_version: "encryptionkey@0"
I20260812 11:20:17.309213 6679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "145aa71aa23f4e60b739ff9460e4218a"
format_stamp: "Formatted at 2026-08-12 11:20:17 on dist-test-slave-gjrz"
server_key: "aa1bf2c44b846a13a3104e5d2fcee6dd"
server_key_iv: "291f7aed13a8888127bd892261bd483c"
server_key_version: "encryptionkey@0"
I20260812 11:20:17.312587 6679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:17.314692 6720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:17.315805 6679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:17.315910 6679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "145aa71aa23f4e60b739ff9460e4218a"
format_stamp: "Formatted at 2026-08-12 11:20:17 on dist-test-slave-gjrz"
server_key: "aa1bf2c44b846a13a3104e5d2fcee6dd"
server_key_iv: "291f7aed13a8888127bd892261bd483c"
server_key_version: "encryptionkey@0"
I20260812 11:20:17.315994 6679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:17.328275 6679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:17.330649 6679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:17.330806 6679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:17.337136 6679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35765
I20260812 11:20:17.337157 6828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35765 every 8 connection(s)
I20260812 11:20:17.337973 6679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:17.340579 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6679
I20260812 11:20:17.340622 6830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:17.340693 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:17.340969 31742 external_mini_cluster.cc:1428] Setting key 8031d8ee61ae4039893a647705e4ccf7
I20260812 11:20:17.347611 6830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a: Bootstrap starting.
I20260812 11:20:17.349978 6830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a: Neither blocks nor log segments found. Creating new log.
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:20:17.350723 6830 log.cc:824] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:17.352994 6830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a: No bootstrap required, opened a new log
I20260812 11:20:17.355844 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:17.342279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48645 (local address 127.0.0.1:35765)
0812 11:20:17.342712 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:17.342732 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:17.342776 (+ 44us) server_negotiation.cc:407] Connection header received
0812 11:20:17.343864 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:17.343891 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:17.344325 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:17.344698 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:17.345334 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:17.346836 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:17.347693 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:17.348355 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:17.348505 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:17.350830 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:17.350837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:17.350853 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:17.350885 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:17.352773 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:17.353306 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:17.353313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:17.353319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:17.353463 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:17.353733 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:17.353737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:17.353738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:17.354217 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:17.354431 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:17.354644 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 11:20:17.354872 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":109,"threads_started":1}
I20260812 11:20:17.357307 6830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } }
I20260812 11:20:17.357525 6830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:17.357585 6830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145aa71aa23f4e60b739ff9460e4218a, State: Initialized, Role: FOLLOWER
I20260812 11:20:17.357995 6830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } }
I20260812 11:20:17.358162 6830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:17.358237 6830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:17.358335 6830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:17.358682 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:17.359189 6830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } }
I20260812 11:20:17.359525 6830 leader_election.cc:302] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 145aa71aa23f4e60b739ff9460e4218a; no voters:
I20260812 11:20:17.359798 6830 leader_election.cc:288] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:17.359887 6837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:17.360047 6837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [term 1 LEADER]: Becoming Leader. State: Replica: 145aa71aa23f4e60b739ff9460e4218a, State: Running, Role: LEADER
I20260812 11:20:17.360388 6837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } }
I20260812 11:20:17.361018 6830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:17.362447 6845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 145aa71aa23f4e60b739ff9460e4218a. Latest consensus state: current_term: 1 leader_uuid: "145aa71aa23f4e60b739ff9460e4218a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } } }
I20260812 11:20:17.362576 6845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:17.362939 6861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:17.362902 6844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "145aa71aa23f4e60b739ff9460e4218a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145aa71aa23f4e60b739ff9460e4218a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35765 } } }
I20260812 11:20:17.362993 6844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:17.366168 6861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:17.369253 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:17.360909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48855 (local address 127.0.0.1:35765)
0812 11:20:17.361011 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:17.361015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:17.361028 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:17.361061 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:17.361063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:17.361103 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:17.361181 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:17.361836 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:17.362560 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:17.363592 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:17.363913 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:17.364016 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:17.365448 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:17.365452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:17.365455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:17.365479 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:17.366989 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:17.367554 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:17.367558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:17.367559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:17.367665 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:17.368009 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:17.368013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:17.368014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:17.368222 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:17.368364 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:17.369016 (+ 652us) server_negotiation.cc:299] Negotiation successful
0812 11:20:17.369109 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:20:17.371544 6861 catalog_manager.cc:1347] Generated new cluster ID: a13f1a5ba97d42c7817893401c68b8da
I20260812 11:20:17.371600 6861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:17.387099 6861 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:17.387908 6861 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:17.397156 6861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 145aa71aa23f4e60b739ff9460e4218a: Generated new TSK 0
I20260812 11:20:17.397701 6861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:17.406440 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6679
W20260812 11:20:17.418915 6653 connection.cc:535] client connection to 127.0.0.1:35765 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35765): Connection reset by peer (error 104)
2026-08-12T11:20:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6878](info): set up 2 sockets
Aug 12 11:20:17 dist-test-slave-gjrz krb5kdc[6878](info): commencing operation
krb5kdc: starting...
W20260812 11:20:18.589825 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:18 dist-test-slave-gjrz krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:18.703753 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:18.811785 6902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:18.812002 6902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:18.812040 6902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:18.815882 6902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:18.815930 6902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:18.815948 6902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:18.815963 6902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:18.815976 6902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:18.820935 6902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:18.821801 6902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:18.822721 6902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:18.830579 6913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:18.830690 6918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:18.830627 6911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:18.830610 6902 server_base.cc:1034] running on GCE node
I20260812 11:20:18.831349 6902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:18.832342 6902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:18.833503 6902 hybrid_clock.cc:648] HybridClock initialized: now 1786533618833488 us; error 25 us; skew 500 ppm
Aug 12 11:20:18 dist-test-slave-gjrz krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:18.836023 6902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:18.836810 6902 webserver.cc:460] Webserver started at http://127.0.0.1:36911/ using document root <none> and password file <none>
I20260812 11:20:18.837298 6902 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:18.837363 6902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:18.837538 6902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:18.839470 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "6ae192c883e3475ebfa0e9105f9fd4c3"
format_stamp: "Formatted at 2026-08-12 11:20:18 on dist-test-slave-gjrz"
server_key: "afdc698ebbba6e4770e32aebf4eb9146"
server_key_iv: "7c090d92bcb34621617643656d3e3a2e"
server_key_version: "encryptionkey@0"
I20260812 11:20:18.839989 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "6ae192c883e3475ebfa0e9105f9fd4c3"
format_stamp: "Formatted at 2026-08-12 11:20:18 on dist-test-slave-gjrz"
server_key: "afdc698ebbba6e4770e32aebf4eb9146"
server_key_iv: "7c090d92bcb34621617643656d3e3a2e"
server_key_version: "encryptionkey@0"
I20260812 11:20:18.843406 6902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:20:18.845597 6934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:18.846676 6902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:18.846793 6902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "6ae192c883e3475ebfa0e9105f9fd4c3"
format_stamp: "Formatted at 2026-08-12 11:20:18 on dist-test-slave-gjrz"
server_key: "afdc698ebbba6e4770e32aebf4eb9146"
server_key_iv: "7c090d92bcb34621617643656d3e3a2e"
server_key_version: "encryptionkey@0"
I20260812 11:20:18.846882 6902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:18.862413 6902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:18.865041 6902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:18.865206 6902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:18.871483 6902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37163
I20260812 11:20:18.871520 7039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37163 every 8 connection(s)
I20260812 11:20:18.872304 6902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:18.874850 7044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:18.878950 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 6902
I20260812 11:20:18.879071 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:18.879382 31742 external_mini_cluster.cc:1428] Setting key 85f643a49190446d5ac900c1dec1bb6c
I20260812 11:20:18.880146 7044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3: Bootstrap starting.
I20260812 11:20:18.882658 7044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:18.883526 7044 log.cc:824] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:18.885344 7044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3: No bootstrap required, opened a new log
I20260812 11:20:18.888790 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:18.880626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52941 (local address 127.0.0.1:37163)
0812 11:20:18.881042 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:18.881060 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:18.881104 (+ 44us) server_negotiation.cc:407] Connection header received
0812 11:20:18.881911 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:18.881934 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:18.882325 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:18.882677 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:18.883088 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:18.884387 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:18.885373 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:18.886050 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:18.886218 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:18.886907 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:18.886913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:18.886929 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:18.887414 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:18.887477 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:18.887504 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:18.887637 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:18.887752 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 11:20:18.887904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":108,"threads_started":1}
I20260812 11:20:18.889401 7044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260812 11:20:18.889595 7044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:18.889638 7044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae192c883e3475ebfa0e9105f9fd4c3, State: Initialized, Role: FOLLOWER
I20260812 11:20:18.890054 7044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260812 11:20:18.890179 7044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:18.890244 7044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:18.890340 7044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:18.891237 7044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260812 11:20:18.891460 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:18.891572 7044 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae192c883e3475ebfa0e9105f9fd4c3; no voters:
I20260812 11:20:18.891855 7044 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:18.891950 7055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:18.892168 7055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae192c883e3475ebfa0e9105f9fd4c3, State: Running, Role: LEADER
I20260812 11:20:18.892509 7055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260812 11:20:18.892995 7044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:18.893682 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae192c883e3475ebfa0e9105f9fd4c3. Latest consensus state: current_term: 1 leader_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260812 11:20:18.893854 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:18.894145 7056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae192c883e3475ebfa0e9105f9fd4c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260812 11:20:18.894239 7056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:18.894304 7080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:18.897207 7080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:18.898327 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:18.893576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34593 (local address 127.0.0.1:37163)
0812 11:20:18.893674 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:18.893678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:18.894011 (+ 333us) server_negotiation.cc:407] Connection header received
0812 11:20:18.894202 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:18.894205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:18.894247 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:18.894293 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:18.894690 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:18.895423 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:18.896536 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:18.896884 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:18.896961 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:18.897917 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:18.897920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:18.897923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:18.897935 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:18.897958 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:18.897968 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:18.898033 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:18.898180 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 11:20:18.898209 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:20:18.902485 7080 catalog_manager.cc:1347] Generated new cluster ID: 3f11a50482f544fd923c60432d0f729d
I20260812 11:20:18.902546 7080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:18.931454 7080 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:18.932282 7080 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:18.941115 7080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae192c883e3475ebfa0e9105f9fd4c3: Generated new TSK 0
I20260812 11:20:18.941844 7080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:18.952791 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 6902
W20260812 11:20:18.970767 6876 connection.cc:535] client connection to 127.0.0.1:37163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37163): Connection reset by peer (error 104)
2026-08-12T11:20:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:19 dist-test-slave-gjrz krb5kdc[7108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:19 dist-test-slave-gjrz krb5kdc[7108](info): set up 2 sockets
Aug 12 11:20:19 dist-test-slave-gjrz krb5kdc[7108](info): commencing operation
krb5kdc: starting...
W20260812 11:20:20.153198 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:20 dist-test-slave-gjrz krb5kdc[7108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:20.271108 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:20.377734 7131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:20.377956 7131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:20.377998 7131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:20.381840 7131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:20.381901 7131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:20.381935 7131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:20.381963 7131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:20.381992 7131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:20.386930 7131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42361
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:20.387826 7131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:20.388734 7131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:20.396515 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:20.396592 7141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:20.396690 7131 server_base.cc:1034] running on GCE node
W20260812 11:20:20.396521 7144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:20.397307 7131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:20.398281 7131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:20.399441 7131 hybrid_clock.cc:648] HybridClock initialized: now 1786533620399424 us; error 28 us; skew 500 ppm
Aug 12 11:20:20 dist-test-slave-gjrz krb5kdc[7108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:20.407094 7131 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:20.407892 7131 webserver.cc:460] Webserver started at http://127.0.0.1:38091/ using document root <none> and password file <none>
I20260812 11:20:20.408365 7131 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:20.408418 7131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:20.408572 7131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:20.410369 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "5dc596597c784ebbb8b76b03ef160aa6"
format_stamp: "Formatted at 2026-08-12 11:20:20 on dist-test-slave-gjrz"
server_key: "e4626c769f666b9d8b12f0cae1ee5347"
server_key_iv: "650987990bea3d655f54f86f16f71142"
server_key_version: "encryptionkey@0"
I20260812 11:20:20.410851 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "5dc596597c784ebbb8b76b03ef160aa6"
format_stamp: "Formatted at 2026-08-12 11:20:20 on dist-test-slave-gjrz"
server_key: "e4626c769f666b9d8b12f0cae1ee5347"
server_key_iv: "650987990bea3d655f54f86f16f71142"
server_key_version: "encryptionkey@0"
I20260812 11:20:20.414219 7131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:20:20.416433 7163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:20.417554 7131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:20.417649 7131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "5dc596597c784ebbb8b76b03ef160aa6"
format_stamp: "Formatted at 2026-08-12 11:20:20 on dist-test-slave-gjrz"
server_key: "e4626c769f666b9d8b12f0cae1ee5347"
server_key_iv: "650987990bea3d655f54f86f16f71142"
server_key_version: "encryptionkey@0"
I20260812 11:20:20.417726 7131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:20.421907 7131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:20.424398 7131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:20.424528 7131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:20.430692 7131 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42361
I20260812 11:20:20.430712 7258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42361 every 8 connection(s)
I20260812 11:20:20.431531 7131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:20.434074 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:20.436236 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7131
I20260812 11:20:20.436381 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:20.436698 31742 external_mini_cluster.cc:1428] Setting key ce48465cb54c41b7a138dae0cbc4796d
I20260812 11:20:20.439895 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: Bootstrap starting.
I20260812 11:20:20.441355 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:20.437922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34183 (local address 127.0.0.1:42361)
0812 11:20:20.438295 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:20.438310 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:20.438343 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:20.438949 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:20.438964 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:20.439228 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:20.439520 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:20.440259 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:20.440264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:20.440274 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:20.440284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:20.440332 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:20.440350 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:20.440447 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:20.440562 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 11:20:20.440694 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":108,"threads_started":1}
I20260812 11:20:20.442298 7259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:20.443053 7259 log.cc:824] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:20.444028 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:20.444911 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: No bootstrap required, opened a new log
I20260812 11:20:20.447247 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:20.445912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33427 (local address 127.0.0.1:42361)
0812 11:20:20.446013 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:20.446016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:20.446167 (+ 151us) server_negotiation.cc:407] Connection header received
0812 11:20:20.446274 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:20.446276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:20.446302 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:20.446360 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:20.446948 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:20.446951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:20.446953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:20.446962 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:20.446983 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:20.446992 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:20.447060 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:20.447131 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 11:20:20.447159 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:20.448773 7259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } }
I20260812 11:20:20.448977 7259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:20.449033 7259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc596597c784ebbb8b76b03ef160aa6, State: Initialized, Role: FOLLOWER
I20260812 11:20:20.449432 7259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } }
I20260812 11:20:20.449561 7259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:20.449635 7259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:20.449728 7259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:20.450524 7259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } }
I20260812 11:20:20.450903 7259 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc596597c784ebbb8b76b03ef160aa6; no voters:
I20260812 11:20:20.451182 7259 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:20.451310 7278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:20.451514 7278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc596597c784ebbb8b76b03ef160aa6, State: Running, Role: LEADER
I20260812 11:20:20.451843 7278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } }
I20260812 11:20:20.452319 7259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:20.453388 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc596597c784ebbb8b76b03ef160aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } } }
I20260812 11:20:20.453510 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:20.453658 7280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc596597c784ebbb8b76b03ef160aa6. Latest consensus state: current_term: 1 leader_uuid: "5dc596597c784ebbb8b76b03ef160aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc596597c784ebbb8b76b03ef160aa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42361 } } }
I20260812 11:20:20.453737 7280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:20.456755 7307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:20.456818 7307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:20.456924 7291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:20.457834 7291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:20.463128 7291 catalog_manager.cc:1347] Generated new cluster ID: b32b3172cb594319be9055dbbe1140d7
I20260812 11:20:20.463196 7291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:20.473140 7291 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:20.473389 7291 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:20.479027 7291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc596597c784ebbb8b76b03ef160aa6: Generated new TSK 0
I20260812 11:20:20.479629 7291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:20.487290 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7131
2026-08-12T11:20:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:20 dist-test-slave-gjrz krb5kdc[7318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:20 dist-test-slave-gjrz krb5kdc[7318](info): set up 2 sockets
Aug 12 11:20:20 dist-test-slave-gjrz krb5kdc[7318](info): commencing operation
krb5kdc: starting...
W20260812 11:20:21.687718 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:21 dist-test-slave-gjrz krb5kdc[7318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:21.801817 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:21.906512 7345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:21.906733 7345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:21.906788 7345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:21.910646 7345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:21.910697 7345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:21.910724 7345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:21.910746 7345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:21.910774 7345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:21.915764 7345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:21.916616 7345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:21.917493 7345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:21.925236 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:21.925228 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:21.925215 7359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:21.925407 7345 server_base.cc:1034] running on GCE node
I20260812 11:20:21.925973 7345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:21.926968 7345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:21.928129 7345 hybrid_clock.cc:648] HybridClock initialized: now 1786533621928110 us; error 26 us; skew 500 ppm
Aug 12 11:20:21 dist-test-slave-gjrz krb5kdc[7318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:21.930683 7345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:21.931509 7345 webserver.cc:460] Webserver started at http://127.0.0.1:39901/ using document root <none> and password file <none>
I20260812 11:20:21.932013 7345 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:21.932078 7345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:21.932251 7345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:21.934113 7345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "a6a99237554f4c089caebd5d9e74b794"
format_stamp: "Formatted at 2026-08-12 11:20:21 on dist-test-slave-gjrz"
server_key: "c040986b6922daaf42374aab47ee6965"
server_key_iv: "f7b2e474cb2170e3ad5fd37dd4b6806f"
server_key_version: "encryptionkey@0"
I20260812 11:20:21.934633 7345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "a6a99237554f4c089caebd5d9e74b794"
format_stamp: "Formatted at 2026-08-12 11:20:21 on dist-test-slave-gjrz"
server_key: "c040986b6922daaf42374aab47ee6965"
server_key_iv: "f7b2e474cb2170e3ad5fd37dd4b6806f"
server_key_version: "encryptionkey@0"
I20260812 11:20:21.938236 7345 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:21.940526 7375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:21.941635 7345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:21.941748 7345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "a6a99237554f4c089caebd5d9e74b794"
format_stamp: "Formatted at 2026-08-12 11:20:21 on dist-test-slave-gjrz"
server_key: "c040986b6922daaf42374aab47ee6965"
server_key_iv: "f7b2e474cb2170e3ad5fd37dd4b6806f"
server_key_version: "encryptionkey@0"
I20260812 11:20:21.941838 7345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:21.954193 7345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:21.957046 7345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:21.957207 7345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:21.963519 7345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37461
I20260812 11:20:21.963550 7473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37461 every 8 connection(s)
I20260812 11:20:21.964367 7345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:21.966663 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7345
I20260812 11:20:21.966775 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:21.966884 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:21.967036 31742 external_mini_cluster.cc:1428] Setting key ea6ab2414308f085681d60816dc4434f
I20260812 11:20:21.973496 7475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794: Bootstrap starting.
I20260812 11:20:21.975744 7475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794: Neither blocks nor log segments found. Creating new log.
Aug 12 11:20:21 dist-test-slave-gjrz krb5kdc[7318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:20:21.976601 7475 log.cc:824] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:21.978615 7475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794: No bootstrap required, opened a new log
I20260812 11:20:21.981209 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:21.968300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37543 (local address 127.0.0.1:37461)
0812 11:20:21.968718 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:21.968735 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:21.968775 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:20:21.969459 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:21.969481 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:21.969883 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:21.970244 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:21.970894 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:21.972379 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:21.973220 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:21.973862 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:21.974015 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:21.976540 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:21.976545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:21.976558 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:21.976584 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:21.978327 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:21.978740 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:21.978746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:21.978751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:21.978886 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:21.979132 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:21.979135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:21.979137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:21.979621 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:21.979794 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:21.980027 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 11:20:21.980308 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":113,"threads_started":1}
I20260812 11:20:21.982774 7475 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260812 11:20:21.983023 7475 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:21.983084 7475 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a99237554f4c089caebd5d9e74b794, State: Initialized, Role: FOLLOWER
I20260812 11:20:21.983516 7475 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260812 11:20:21.983654 7475 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:21.983718 7475 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:21.983814 7475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:21.983826 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:21.984556 7475 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260812 11:20:21.984884 7475 leader_election.cc:302] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6a99237554f4c089caebd5d9e74b794; no voters:
I20260812 11:20:21.985190 7475 leader_election.cc:288] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:21.985275 7497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:21.985460 7497 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [term 1 LEADER]: Becoming Leader. State: Replica: a6a99237554f4c089caebd5d9e74b794, State: Running, Role: LEADER
I20260812 11:20:21.985752 7497 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260812 11:20:21.986367 7475 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:21.987187 7497 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6a99237554f4c089caebd5d9e74b794. Latest consensus state: current_term: 1 leader_uuid: "a6a99237554f4c089caebd5d9e74b794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260812 11:20:21.987391 7497 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:21.987447 7498 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6a99237554f4c089caebd5d9e74b794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a99237554f4c089caebd5d9e74b794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260812 11:20:21.987584 7498 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:21.987821 7517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:21.990772 7517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:21.993649 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:21.985896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52583 (local address 127.0.0.1:37461)
0812 11:20:21.985988 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:21.985992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:21.986007 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:20:21.986131 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:21.986134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:21.986177 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:21.986238 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:21.986950 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:21.987708 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:21.988798 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:21.989036 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:21.989132 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:21.990571 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:21.990574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:21.990577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:21.990607 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:21.992014 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:21.992394 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:21.992398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:21.992400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:21.992477 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:21.992732 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:21.992736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:21.992737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:21.992933 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:21.993047 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:21.993392 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 11:20:21.993498 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 11:20:21.997048 7517 catalog_manager.cc:1347] Generated new cluster ID: 1ffa55ee4735457e8d10a23a2ce1d532
I20260812 11:20:21.997099 7517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:22.010710 7517 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:22.011874 7517 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:22.027485 7517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6a99237554f4c089caebd5d9e74b794: Generated new TSK 0
I20260812 11:20:22.028211 7517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:22.035360 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7345
2026-08-12T11:20:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:22 dist-test-slave-gjrz krb5kdc[7554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:22 dist-test-slave-gjrz krb5kdc[7554](info): set up 2 sockets
Aug 12 11:20:22 dist-test-slave-gjrz krb5kdc[7554](info): commencing operation
krb5kdc: starting...
W20260812 11:20:23.248095 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:23 dist-test-slave-gjrz krb5kdc[7554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:23.361814 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:23.466962 7590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:23.467188 7590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:23.467226 7590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:23.471043 7590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:23.471091 7590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:23.471109 7590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:23.471124 7590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:23.471138 7590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:23.476080 7590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:23.476920 7590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:23.477840 7590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:23.485800 7605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:23.485785 7600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:23.485976 7590 server_base.cc:1034] running on GCE node
W20260812 11:20:23.485824 7601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:23.486574 7590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:23.487658 7590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:23.488816 7590 hybrid_clock.cc:648] HybridClock initialized: now 1786533623488799 us; error 25 us; skew 500 ppm
Aug 12 11:20:23 dist-test-slave-gjrz krb5kdc[7554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:23.491514 7590 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:23.492467 7590 webserver.cc:460] Webserver started at http://127.0.0.1:33445/ using document root <none> and password file <none>
I20260812 11:20:23.493114 7590 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:23.493196 7590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:23.493434 7590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:23.495982 7590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "d71037796fbc4ab3a9c748fc6dc904c5"
format_stamp: "Formatted at 2026-08-12 11:20:23 on dist-test-slave-gjrz"
server_key: "d394ec9000b84913f0cdcf8220399cb4"
server_key_iv: "e3150ddb8d28468717af5c9aa4775572"
server_key_version: "encryptionkey@0"
I20260812 11:20:23.496654 7590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "d71037796fbc4ab3a9c748fc6dc904c5"
format_stamp: "Formatted at 2026-08-12 11:20:23 on dist-test-slave-gjrz"
server_key: "d394ec9000b84913f0cdcf8220399cb4"
server_key_iv: "e3150ddb8d28468717af5c9aa4775572"
server_key_version: "encryptionkey@0"
I20260812 11:20:23.500288 7590 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 11:20:23.502462 7612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:23.503602 7590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:23.503708 7590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "d71037796fbc4ab3a9c748fc6dc904c5"
format_stamp: "Formatted at 2026-08-12 11:20:23 on dist-test-slave-gjrz"
server_key: "d394ec9000b84913f0cdcf8220399cb4"
server_key_iv: "e3150ddb8d28468717af5c9aa4775572"
server_key_version: "encryptionkey@0"
I20260812 11:20:23.503786 7590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:23.526675 7590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:23.529495 7590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:23.529642 7590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:23.536137 7590 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38799
I20260812 11:20:23.536151 7731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38799 every 8 connection(s)
I20260812 11:20:23.537032 7590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:23.539597 7732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:23.544713 7732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: Bootstrap starting.
I20260812 11:20:23.546931 7732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:23.546967 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7590
I20260812 11:20:23.547092 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:23.547382 31742 external_mini_cluster.cc:1428] Setting key f9bec6ba2a926339dae7e5a80a13b69e
I20260812 11:20:23.547708 7732 log.cc:824] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:23.549646 7732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: No bootstrap required, opened a new log
I20260812 11:20:23.552821 7732 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260812 11:20:23.553020 7732 raft_consensus.cc:374] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:23.553112 7732 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71037796fbc4ab3a9c748fc6dc904c5, State: Initialized, Role: FOLLOWER
I20260812 11:20:23.553525 7732 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260812 11:20:23.553658 7732 raft_consensus.cc:388] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:23.553721 7732 raft_consensus.cc:482] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:23.553814 7732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:23.554589 7732 raft_consensus.cc:504] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260812 11:20:23.554903 7732 leader_election.cc:302] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d71037796fbc4ab3a9c748fc6dc904c5; no voters:
I20260812 11:20:23.555168 7732 leader_election.cc:288] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:23.555291 7742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:23.555390 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:23.548386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40505 (local address 127.0.0.1:38799)
0812 11:20:23.548741 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:23.548755 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:23.548789 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:20:23.549387 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:23.549404 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:23.549720 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:23.550033 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:23.550425 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:23.551442 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:23.552333 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:23.552800 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:23.552947 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:23.553708 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:23.553715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:23.553731 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:23.554131 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:23.554183 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:23.554207 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:23.554291 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:23.554429 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:20:23.554620 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":107,"threads_started":1}
I20260812 11:20:23.555465 7742 raft_consensus.cc:686] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [term 1 LEADER]: Becoming Leader. State: Replica: d71037796fbc4ab3a9c748fc6dc904c5, State: Running, Role: LEADER
I20260812 11:20:23.555881 7742 consensus_queue.cc:237] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } }
I20260812 11:20:23.556335 7732 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:23.558009 7745 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d71037796fbc4ab3a9c748fc6dc904c5. Latest consensus state: current_term: 1 leader_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } } }
I20260812 11:20:23.558125 7743 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71037796fbc4ab3a9c748fc6dc904c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38799 } } }
I20260812 11:20:23.558215 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:23.558136 7745 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:23.558264 7743 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:23.561343 7768 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:23.561466 7768 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:23.561561 7769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:23.562271 7769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:23.564376 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:23.560251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44747 (local address 127.0.0.1:38799)
0812 11:20:23.560471 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:23.560475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:23.560492 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:20:23.560627 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:23.560630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:23.560671 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:23.560750 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:23.561001 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:23.561691 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:23.562803 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:23.563112 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:23.563215 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:23.563988 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:23.563992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:23.563995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:23.564008 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:23.564035 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:23.564048 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:23.564123 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:23.564217 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 11:20:23.564262 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260812 11:20:23.567337 7769 catalog_manager.cc:1347] Generated new cluster ID: 0ec158d33a8549d5887ea4ead8c8542d
I20260812 11:20:23.567390 7769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:23.587091 7769 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:23.587941 7769 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:23.593909 7769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d71037796fbc4ab3a9c748fc6dc904c5: Generated new TSK 0
I20260812 11:20:23.594473 7769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:23.601280 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7590
2026-08-12T11:20:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:23 dist-test-slave-gjrz krb5kdc[7795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:23 dist-test-slave-gjrz krb5kdc[7795](info): set up 2 sockets
Aug 12 11:20:23 dist-test-slave-gjrz krb5kdc[7795](info): commencing operation
krb5kdc: starting...
W20260812 11:20:24.796880 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:24 dist-test-slave-gjrz krb5kdc[7795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:24.909760 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:25.016975 7814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:25.017195 7814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:25.017233 7814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:25.021076 7814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:25.021122 7814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:25.021140 7814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:25.021160 7814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:25.021179 7814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:25.026077 7814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:25.026934 7814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:25.027884 7814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:25.035588 7827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:25.035656 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:25.035652 7830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:25.035825 7814 server_base.cc:1034] running on GCE node
I20260812 11:20:25.036420 7814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:25.037434 7814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:25.038606 7814 hybrid_clock.cc:648] HybridClock initialized: now 1786533625038589 us; error 26 us; skew 500 ppm
Aug 12 11:20:25 dist-test-slave-gjrz krb5kdc[7795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:25.041225 7814 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:25.042042 7814 webserver.cc:460] Webserver started at http://127.0.0.1:35353/ using document root <none> and password file <none>
I20260812 11:20:25.042551 7814 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:25.042618 7814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:25.042800 7814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:25.044809 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "c09113fd66584a45ad36db4261a2db11"
format_stamp: "Formatted at 2026-08-12 11:20:25 on dist-test-slave-gjrz"
server_key: "ad36439e4342a2e264b98d4d3dee137c"
server_key_iv: "96df7a6cc56e3edc20fadda75f6f9714"
server_key_version: "encryptionkey@0"
I20260812 11:20:25.045342 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "c09113fd66584a45ad36db4261a2db11"
format_stamp: "Formatted at 2026-08-12 11:20:25 on dist-test-slave-gjrz"
server_key: "ad36439e4342a2e264b98d4d3dee137c"
server_key_iv: "96df7a6cc56e3edc20fadda75f6f9714"
server_key_version: "encryptionkey@0"
I20260812 11:20:25.048807 7814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:25.050987 7847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:25.052132 7814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:25.052245 7814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "c09113fd66584a45ad36db4261a2db11"
format_stamp: "Formatted at 2026-08-12 11:20:25 on dist-test-slave-gjrz"
server_key: "ad36439e4342a2e264b98d4d3dee137c"
server_key_iv: "96df7a6cc56e3edc20fadda75f6f9714"
server_key_version: "encryptionkey@0"
I20260812 11:20:25.052335 7814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:25.056623 7814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:25.059087 7814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:25.059252 7814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:25.065783 7814 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42487
I20260812 11:20:25.065831 7941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42487 every 8 connection(s)
I20260812 11:20:25.066602 7814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:25.069159 7942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:25.074142 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: Bootstrap starting.
I20260812 11:20:25.074538 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 7814
I20260812 11:20:25.074661 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:25.074882 31742 external_mini_cluster.cc:1428] Setting key 871c69b4696888c84e93a76717c43956
I20260812 11:20:25.076500 7942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:25.077342 7942 log.cc:824] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:25.079326 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: No bootstrap required, opened a new log
I20260812 11:20:25.079911 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:25.076091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56065 (local address 127.0.0.1:42487)
0812 11:20:25.076526 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:25.076544 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:25.076586 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:20:25.077390 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:25.077414 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:25.077754 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:25.078059 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:25.078757 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:25.078761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:25.078772 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:25.078782 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:25.078832 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:25.078850 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:25.078940 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:25.079060 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 11:20:25.079206 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260812 11:20:25.082535 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:25.083091 7942 raft_consensus.cc:348] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } }
I20260812 11:20:25.083326 7942 raft_consensus.cc:374] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:25.083384 7942 raft_consensus.cc:729] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09113fd66584a45ad36db4261a2db11, State: Initialized, Role: FOLLOWER
I20260812 11:20:25.083787 7942 consensus_queue.cc:260] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } }
I20260812 11:20:25.083940 7942 raft_consensus.cc:388] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:25.084003 7942 raft_consensus.cc:482] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:25.084138 7942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:25.085052 7942 raft_consensus.cc:504] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } }
I20260812 11:20:25.085402 7942 leader_election.cc:302] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c09113fd66584a45ad36db4261a2db11; no voters:
I20260812 11:20:25.085721 7942 leader_election.cc:288] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:25.085800 7959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:25.086109 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:25.084525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42985 (local address 127.0.0.1:42487)
0812 11:20:25.084621 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:25.084624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:25.084802 (+ 178us) server_negotiation.cc:407] Connection header received
0812 11:20:25.084913 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:25.084915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:25.084944 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:25.085001 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:25.085593 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:25.085597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:25.085599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:25.085633 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:25.085655 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:25.085664 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:25.085731 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:25.085953 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 11:20:25.085989 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:20:25.086091 7959 raft_consensus.cc:686] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [term 1 LEADER]: Becoming Leader. State: Replica: c09113fd66584a45ad36db4261a2db11, State: Running, Role: LEADER
I20260812 11:20:25.086470 7959 consensus_queue.cc:237] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } }
I20260812 11:20:25.086933 7942 sys_catalog.cc:564] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:25.088217 7965 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c09113fd66584a45ad36db4261a2db11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } } }
I20260812 11:20:25.088250 7966 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c09113fd66584a45ad36db4261a2db11. Latest consensus state: current_term: 1 leader_uuid: "c09113fd66584a45ad36db4261a2db11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09113fd66584a45ad36db4261a2db11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42487 } } }
I20260812 11:20:25.088337 7965 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:25.088347 7966 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:25.091714 7987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:25.091781 7987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:25.091857 7982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:25.092518 7982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:25.097621 7982 catalog_manager.cc:1347] Generated new cluster ID: 4a516369389c4b6eb7f4d39641bf1e1e
I20260812 11:20:25.097681 7982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:25.125212 7982 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:25.125499 7982 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:25.130774 7982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c09113fd66584a45ad36db4261a2db11: Generated new TSK 0
I20260812 11:20:25.131415 7982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:25.135000 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 7814
2026-08-12T11:20:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:25 dist-test-slave-gjrz krb5kdc[7999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:25 dist-test-slave-gjrz krb5kdc[7999](info): set up 2 sockets
Aug 12 11:20:25 dist-test-slave-gjrz krb5kdc[7999](info): commencing operation
krb5kdc: starting...
W20260812 11:20:26.352247 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[7999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:26.464201 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:26.570076 8031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:26.570298 8031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:26.570353 8031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:26.574218 8031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:26.574267 8031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:26.574293 8031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:26.574316 8031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:26.574344 8031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:26.579280 8031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:26.580207 8031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:26.581122 8031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:26.588908 8044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:26.588910 8042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:26.588944 8040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:26.589200 8031 server_base.cc:1034] running on GCE node
I20260812 11:20:26.589725 8031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:26.590690 8031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:26.591846 8031 hybrid_clock.cc:648] HybridClock initialized: now 1786533626591825 us; error 39 us; skew 500 ppm
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[7999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:26.594295 8031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:26.595064 8031 webserver.cc:460] Webserver started at http://127.0.0.1:33299/ using document root <none> and password file <none>
I20260812 11:20:26.595595 8031 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:26.595661 8031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:26.595849 8031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:26.597700 8031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "a372a30f0d964246bbca1e6f743fb138"
format_stamp: "Formatted at 2026-08-12 11:20:26 on dist-test-slave-gjrz"
server_key: "96976467987259bde5957cd05dd724ae"
server_key_iv: "ba6c46eb3b8bea09172ddba4a77db44a"
server_key_version: "encryptionkey@0"
I20260812 11:20:26.598222 8031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "a372a30f0d964246bbca1e6f743fb138"
format_stamp: "Formatted at 2026-08-12 11:20:26 on dist-test-slave-gjrz"
server_key: "96976467987259bde5957cd05dd724ae"
server_key_iv: "ba6c46eb3b8bea09172ddba4a77db44a"
server_key_version: "encryptionkey@0"
I20260812 11:20:26.601611 8031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:26.603760 8057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:26.604837 8031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:26.604951 8031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "a372a30f0d964246bbca1e6f743fb138"
format_stamp: "Formatted at 2026-08-12 11:20:26 on dist-test-slave-gjrz"
server_key: "96976467987259bde5957cd05dd724ae"
server_key_iv: "ba6c46eb3b8bea09172ddba4a77db44a"
server_key_version: "encryptionkey@0"
I20260812 11:20:26.605038 8031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:26.628806 8031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:26.631328 8031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:26.631491 8031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:26.637863 8031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33661
I20260812 11:20:26.637892 8173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33661 every 8 connection(s)
I20260812 11:20:26.638715 8031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:26.639111 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8031
I20260812 11:20:26.639254 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:26.639559 31742 external_mini_cluster.cc:1428] Setting key bcbd4e4db2587397cfbf56fa77fd0e84
I20260812 11:20:26.641605 8179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:26.647821 8179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: Bootstrap starting.
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[7999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:20:26.650003 8179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:26.650828 8179 log.cc:824] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:26.652686 8179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: No bootstrap required, opened a new log
I20260812 11:20:26.652976 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:26.640789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:51903 (local address 127.0.0.1:33661)
0812 11:20:26.641175 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:26.641189 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:26.641223 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:20:26.641844 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:26.641861 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:26.642189 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:26.642445 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:26.643334 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:26.644284 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:26.645432 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:26.645881 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:26.645986 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:26.648677 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:26.648682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:26.648693 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:26.648719 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:26.650218 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:26.650741 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:26.650746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:26.650750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:26.650852 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:26.651184 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:26.651186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:26.651188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:26.651581 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:26.651767 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:26.652095 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 11:20:26.652285 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
I20260812 11:20:26.656178 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:26.656116 8179 raft_consensus.cc:348] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } }
I20260812 11:20:26.656366 8179 raft_consensus.cc:374] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:26.656417 8179 raft_consensus.cc:729] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a372a30f0d964246bbca1e6f743fb138, State: Initialized, Role: FOLLOWER
I20260812 11:20:26.656858 8179 consensus_queue.cc:260] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } }
I20260812 11:20:26.657022 8179 raft_consensus.cc:388] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:26.657089 8179 raft_consensus.cc:482] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:26.657186 8179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:26.658236 8179 raft_consensus.cc:504] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } }
I20260812 11:20:26.658586 8179 leader_election.cc:302] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a372a30f0d964246bbca1e6f743fb138; no voters:
I20260812 11:20:26.658926 8179 leader_election.cc:288] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:26.659029 8197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:26.659243 8197 raft_consensus.cc:686] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [term 1 LEADER]: Becoming Leader. State: Replica: a372a30f0d964246bbca1e6f743fb138, State: Running, Role: LEADER
I20260812 11:20:26.659641 8197 consensus_queue.cc:237] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } }
I20260812 11:20:26.660121 8179 sys_catalog.cc:564] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:26.661132 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a372a30f0d964246bbca1e6f743fb138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } } }
I20260812 11:20:26.661258 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:26.661563 8199 sys_catalog.cc:455] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a372a30f0d964246bbca1e6f743fb138. Latest consensus state: current_term: 1 leader_uuid: "a372a30f0d964246bbca1e6f743fb138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372a30f0d964246bbca1e6f743fb138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33661 } } }
I20260812 11:20:26.661731 8199 sys_catalog.cc:458] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:26.665126 8226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:26.665198 8226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:26.665258 8209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:26.666043 8209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:26.666872 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:26.658343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52839 (local address 127.0.0.1:33661)
0812 11:20:26.658444 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:26.658448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:26.658578 (+ 130us) server_negotiation.cc:407] Connection header received
0812 11:20:26.658656 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:26.658658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:26.658703 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:26.658781 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:26.659546 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:26.660256 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:26.661417 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:26.661733 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:26.661826 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:26.663405 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:26.663408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:26.663410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:26.663431 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:26.664879 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:26.665425 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:26.665429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:26.665431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:26.665520 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:26.665885 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:26.665888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:26.665890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:26.666168 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:26.666312 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:26.666633 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 11:20:26.666729 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:26.671003 8209 catalog_manager.cc:1347] Generated new cluster ID: 3b0155221498412397d7930b976883e1
I20260812 11:20:26.671053 8209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:26.681027 8209 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:26.681800 8209 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:26.686127 8209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a372a30f0d964246bbca1e6f743fb138: Generated new TSK 0
I20260812 11:20:26.686662 8209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:26.690039 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8031
W20260812 11:20:26.708787 7996 connection.cc:535] client connection to 127.0.0.1:33661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33661): Connection reset by peer (error 104)
2026-08-12T11:20:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[8236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[8236](info): set up 2 sockets
Aug 12 11:20:26 dist-test-slave-gjrz krb5kdc[8236](info): commencing operation
krb5kdc: starting...
W20260812 11:20:27.904670 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:27 dist-test-slave-gjrz krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:28.019388 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:28.126968 8272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:28.127200 8272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:28.127256 8272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:28.131070 8272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:28.131122 8272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:28.131148 8272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:28.131170 8272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:28.131198 8272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:28.136186 8272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:28.137066 8272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:28.137951 8272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:28.145823 8283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:28.145820 8282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:28.145820 8287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:28.146173 8272 server_base.cc:1034] running on GCE node
I20260812 11:20:28.146679 8272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:28.147709 8272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:28.148861 8272 hybrid_clock.cc:648] HybridClock initialized: now 1786533628148847 us; error 28 us; skew 500 ppm
Aug 12 11:20:28 dist-test-slave-gjrz krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:28.151420 8272 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:28.152230 8272 webserver.cc:460] Webserver started at http://127.0.0.1:44409/ using document root <none> and password file <none>
I20260812 11:20:28.152719 8272 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:28.152781 8272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:28.152968 8272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:28.154865 8272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "b0490877444147a08c3fc97604923004"
format_stamp: "Formatted at 2026-08-12 11:20:28 on dist-test-slave-gjrz"
server_key: "1c5190345a92fdc7430180b72794fa19"
server_key_iv: "71a1f4e970a214f09f90411d5d5ca985"
server_key_version: "encryptionkey@0"
I20260812 11:20:28.155416 8272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "b0490877444147a08c3fc97604923004"
format_stamp: "Formatted at 2026-08-12 11:20:28 on dist-test-slave-gjrz"
server_key: "1c5190345a92fdc7430180b72794fa19"
server_key_iv: "71a1f4e970a214f09f90411d5d5ca985"
server_key_version: "encryptionkey@0"
I20260812 11:20:28.158789 8272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:28.161001 8299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:28.162106 8272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:28.162212 8272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "b0490877444147a08c3fc97604923004"
format_stamp: "Formatted at 2026-08-12 11:20:28 on dist-test-slave-gjrz"
server_key: "1c5190345a92fdc7430180b72794fa19"
server_key_iv: "71a1f4e970a214f09f90411d5d5ca985"
server_key_version: "encryptionkey@0"
I20260812 11:20:28.162297 8272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:28.200381 8272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:28.203382 8272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:28.203552 8272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:28.210278 8272 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33839
I20260812 11:20:28.210294 8396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33839 every 8 connection(s)
I20260812 11:20:28.211170 8272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:28.213840 8402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:28.215015 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8272
I20260812 11:20:28.215143 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:28.215479 31742 external_mini_cluster.cc:1428] Setting key 367bba1e70b8d7ed692baa9d0dbed033
I20260812 11:20:28.220247 8402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004: Bootstrap starting.
I20260812 11:20:28.222716 8402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:28.223565 8402 log.cc:824] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:28.224960 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:28.216765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:33049 (local address 127.0.0.1:33839)
0812 11:20:28.217178 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:28.217196 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:28.217238 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:20:28.218022 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:28.218044 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:28.218475 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:28.218819 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:28.219241 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:28.220644 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:28.221529 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:28.222222 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:28.222383 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:28.223080 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:28.223086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:28.223101 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:28.223602 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:28.223673 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:28.223706 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:28.223836 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:28.223972 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 11:20:28.224104 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260812 11:20:28.225669 8402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004: No bootstrap required, opened a new log
I20260812 11:20:28.227644 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:28.229398 8402 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260812 11:20:28.229674 8402 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:28.229743 8402 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0490877444147a08c3fc97604923004, State: Initialized, Role: FOLLOWER
I20260812 11:20:28.230147 8402 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260812 11:20:28.230278 8402 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:28.230341 8402 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:28.230433 8402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:28.231207 8402 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260812 11:20:28.231541 8402 leader_election.cc:302] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0490877444147a08c3fc97604923004; no voters:
I20260812 11:20:28.231818 8402 leader_election.cc:288] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:28.231902 8429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:28.232077 8429 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [term 1 LEADER]: Becoming Leader. State: Replica: b0490877444147a08c3fc97604923004, State: Running, Role: LEADER
I20260812 11:20:28.232380 8429 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260812 11:20:28.232999 8402 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:28.233173 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:28.229437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52189 (local address 127.0.0.1:33839)
0812 11:20:28.229530 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:28.229534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:28.229800 (+ 266us) server_negotiation.cc:407] Connection header received
0812 11:20:28.229838 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:28.229840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:28.229879 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:28.229952 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:28.230198 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:28.230782 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:28.231636 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:28.231965 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:28.232060 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:28.232782 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:28.232785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:28.232788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:28.232800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:28.232824 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:28.232834 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:28.232905 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:28.233025 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 11:20:28.233054 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:20:28.233511 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0490877444147a08c3fc97604923004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } } }
I20260812 11:20:28.233628 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:28.233865 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0490877444147a08c3fc97604923004. Latest consensus state: current_term: 1 leader_uuid: "b0490877444147a08c3fc97604923004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0490877444147a08c3fc97604923004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } } }
I20260812 11:20:28.233954 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:28.234248 8441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:28.237298 8441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:28.242152 8441 catalog_manager.cc:1347] Generated new cluster ID: c417dc48efc7492baab96548cc2f06de
I20260812 11:20:28.242216 8441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:28.254989 8441 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:28.255816 8441 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:28.263443 8441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0490877444147a08c3fc97604923004: Generated new TSK 0
I20260812 11:20:28.264003 8441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:28.278035 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8272
W20260812 11:20:28.286989 8233 connection.cc:535] client connection to 127.0.0.1:33839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33839): Connection reset by peer (error 104)
2026-08-12T11:20:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:28 dist-test-slave-gjrz krb5kdc[8480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:28 dist-test-slave-gjrz krb5kdc[8480](info): set up 2 sockets
Aug 12 11:20:28 dist-test-slave-gjrz krb5kdc[8480](info): commencing operation
krb5kdc: starting...
W20260812 11:20:29.475991 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:29 dist-test-slave-gjrz krb5kdc[8480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:29.590926 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:29.698405 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:29.698634 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:29.698691 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:29.702656 8502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:29.702709 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:29.702734 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:29.702756 8502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:29.702785 8502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:29.707696 8502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:29.708595 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:29.709496 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:29.717303 8521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:29.717447 8502 server_base.cc:1034] running on GCE node
W20260812 11:20:29.717312 8512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:29.717312 8514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:29.718072 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:29.719045 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:29.720199 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786533629720166 us; error 44 us; skew 500 ppm
Aug 12 11:20:29 dist-test-slave-gjrz krb5kdc[8480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:29.722697 8502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:29.723515 8502 webserver.cc:460] Webserver started at http://127.0.0.1:35045/ using document root <none> and password file <none>
I20260812 11:20:29.723985 8502 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:29.724048 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:29.724207 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:29.726063 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "adc41cf8ff814389b5c471dd9502028f"
format_stamp: "Formatted at 2026-08-12 11:20:29 on dist-test-slave-gjrz"
server_key: "63a0f9e4aa8f864bbe41804a0175b6f9"
server_key_iv: "5afb062d5ac47066368f366e885b0d27"
server_key_version: "encryptionkey@0"
I20260812 11:20:29.726555 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "adc41cf8ff814389b5c471dd9502028f"
format_stamp: "Formatted at 2026-08-12 11:20:29 on dist-test-slave-gjrz"
server_key: "63a0f9e4aa8f864bbe41804a0175b6f9"
server_key_iv: "5afb062d5ac47066368f366e885b0d27"
server_key_version: "encryptionkey@0"
I20260812 11:20:29.730026 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:29.732316 8536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:29.733430 8502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:29.733530 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "adc41cf8ff814389b5c471dd9502028f"
format_stamp: "Formatted at 2026-08-12 11:20:29 on dist-test-slave-gjrz"
server_key: "63a0f9e4aa8f864bbe41804a0175b6f9"
server_key_iv: "5afb062d5ac47066368f366e885b0d27"
server_key_version: "encryptionkey@0"
I20260812 11:20:29.733625 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:29.737845 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:29.740690 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:29.740841 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:29.747288 8502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41051
I20260812 11:20:29.747319 8652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41051 every 8 connection(s)
I20260812 11:20:29.748118 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:29.750671 8654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:29.755597 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8502
I20260812 11:20:29.755723 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:29.755921 8654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f: Bootstrap starting.
I20260812 11:20:29.755959 31742 external_mini_cluster.cc:1428] Setting key 498ad3ce80a5ac61946baa602b5f9cd3
I20260812 11:20:29.758226 8654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:29.759022 8654 log.cc:824] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:29.760646 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:29.757085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:37781 (local address 127.0.0.1:41051)
0812 11:20:29.757470 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:29.757484 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:29.757517 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:29.758112 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:29.758128 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:29.758403 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:29.758675 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:29.759488 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:29.759494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:29.759509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:29.759522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:29.759583 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:29.759612 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:29.759723 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:29.759832 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 11:20:29.759972 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260812 11:20:29.760980 8654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f: No bootstrap required, opened a new log
I20260812 11:20:29.763305 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:29.765197 8654 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260812 11:20:29.765451 8654 raft_consensus.cc:374] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:29.765497 8654 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc41cf8ff814389b5c471dd9502028f, State: Initialized, Role: FOLLOWER
I20260812 11:20:29.765928 8654 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260812 11:20:29.766096 8654 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:29.766170 8654 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:29.766273 8654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:29.766952 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:29.765529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:58799 (local address 127.0.0.1:41051)
0812 11:20:29.765706 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:29.765710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:29.765775 (+ 65us) server_negotiation.cc:407] Connection header received
0812 11:20:29.765914 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:29.765916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:29.765942 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:29.765998 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:29.766649 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:29.766652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:29.766654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:29.766664 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:29.766686 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:29.766696 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:29.766753 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:29.766838 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 11:20:29.766866 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 11:20:29.767324 8654 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260812 11:20:29.767678 8654 leader_election.cc:302] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc41cf8ff814389b5c471dd9502028f; no voters:
I20260812 11:20:29.767988 8654 leader_election.cc:288] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:29.768070 8668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:29.768253 8668 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [term 1 LEADER]: Becoming Leader. State: Replica: adc41cf8ff814389b5c471dd9502028f, State: Running, Role: LEADER
I20260812 11:20:29.768527 8668 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260812 11:20:29.769199 8654 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:29.770277 8670 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc41cf8ff814389b5c471dd9502028f. Latest consensus state: current_term: 1 leader_uuid: "adc41cf8ff814389b5c471dd9502028f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } } }
I20260812 11:20:29.770256 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adc41cf8ff814389b5c471dd9502028f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc41cf8ff814389b5c471dd9502028f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } } }
I20260812 11:20:29.770396 8670 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:29.770396 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:29.770845 8685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:29.773547 8685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:29.778628 8685 catalog_manager.cc:1347] Generated new cluster ID: e9e22b84542c46c8bb056fa4abe0630f
I20260812 11:20:29.778693 8685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:29.795341 8685 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:29.795626 8685 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:29.801391 8685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adc41cf8ff814389b5c471dd9502028f: Generated new TSK 0
I20260812 11:20:29.802021 8685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:29.807547 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8502
2026-08-12T11:20:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:29 dist-test-slave-gjrz krb5kdc[8722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:29 dist-test-slave-gjrz krb5kdc[8722](info): set up 2 sockets
Aug 12 11:20:29 dist-test-slave-gjrz krb5kdc[8722](info): commencing operation
krb5kdc: starting...
W20260812 11:20:30.997984 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:31.111773 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34303
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:31.217419 8744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:31.217643 8744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:31.217680 8744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:31.221510 8744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:31.221560 8744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:31.221580 8744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:31.221607 8744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:31.221635 8744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:31.226547 8744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34303
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:31.227427 8744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:31.228317 8744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:31.236135 8759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:31.236231 8744 server_base.cc:1034] running on GCE node
W20260812 11:20:31.236105 8757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:31.236104 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:31.236913 8744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:31.237866 8744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:31.239019 8744 hybrid_clock.cc:648] HybridClock initialized: now 1786533631238993 us; error 37 us; skew 500 ppm
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533631, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:31.245200 8744 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:31.245990 8744 webserver.cc:460] Webserver started at http://127.0.0.1:37359/ using document root <none> and password file <none>
I20260812 11:20:31.246493 8744 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:31.246567 8744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:31.246759 8744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:31.248641 8744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "847cedabfec648e0bc8f420b2f393db0"
format_stamp: "Formatted at 2026-08-12 11:20:31 on dist-test-slave-gjrz"
server_key: "1a7d73021c4998be9f65d71a25281ce4"
server_key_iv: "f26ecd9f7bd81619ed825a5d1e9cda20"
server_key_version: "encryptionkey@0"
I20260812 11:20:31.249146 8744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "847cedabfec648e0bc8f420b2f393db0"
format_stamp: "Formatted at 2026-08-12 11:20:31 on dist-test-slave-gjrz"
server_key: "1a7d73021c4998be9f65d71a25281ce4"
server_key_iv: "f26ecd9f7bd81619ed825a5d1e9cda20"
server_key_version: "encryptionkey@0"
I20260812 11:20:31.252516 8744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:20:31.254604 8772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:31.255731 8744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:31.255839 8744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "847cedabfec648e0bc8f420b2f393db0"
format_stamp: "Formatted at 2026-08-12 11:20:31 on dist-test-slave-gjrz"
server_key: "1a7d73021c4998be9f65d71a25281ce4"
server_key_iv: "f26ecd9f7bd81619ed825a5d1e9cda20"
server_key_version: "encryptionkey@0"
I20260812 11:20:31.255921 8744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:31.264681 8744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:31.267396 8744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:31.267558 8744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:31.273840 8744 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34303
I20260812 11:20:31.273854 8894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34303 every 8 connection(s)
I20260812 11:20:31.274710 8744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:31.276706 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8744
I20260812 11:20:31.276860 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:31.277170 31742 external_mini_cluster.cc:1428] Setting key 305759283663b294b54ffd300f0236ce
I20260812 11:20:31.277542 8896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:31.283360 8896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: Bootstrap starting.
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:20:31.285601 8896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:31.286293 8896 log.cc:824] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:31.288142 8896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: No bootstrap required, opened a new log
I20260812 11:20:31.290351 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:31.278457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:35533 (local address 127.0.0.1:34303)
0812 11:20:31.278895 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:31.278909 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:31.278942 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:31.279604 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:31.279621 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:31.279927 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:31.280214 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:31.281002 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:31.281984 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:31.282830 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:31.283329 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:31.283463 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:31.285981 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:31.285986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:31.285999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:31.286024 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:31.287609 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:31.288080 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:31.288086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:31.288090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:31.288194 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:31.288610 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:31.288613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:31.288614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:31.288975 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:31.289153 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:31.289467 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 11:20:31.289678 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":175,"threads_started":1}
I20260812 11:20:31.291988 8896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } }
I20260812 11:20:31.292222 8896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:31.292315 8896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cedabfec648e0bc8f420b2f393db0, State: Initialized, Role: FOLLOWER
I20260812 11:20:31.292760 8896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } }
I20260812 11:20:31.292896 8896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:31.292956 8896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:31.293056 8896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:31.293192 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:31.293968 8896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } }
I20260812 11:20:31.294294 8896 leader_election.cc:302] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 847cedabfec648e0bc8f420b2f393db0; no voters:
I20260812 11:20:31.294597 8896 leader_election.cc:288] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:31.294687 8908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:31.294910 8908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [term 1 LEADER]: Becoming Leader. State: Replica: 847cedabfec648e0bc8f420b2f393db0, State: Running, Role: LEADER
I20260812 11:20:31.295493 8908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } }
I20260812 11:20:31.295825 8896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:31.297183 8915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 847cedabfec648e0bc8f420b2f393db0. Latest consensus state: current_term: 1 leader_uuid: "847cedabfec648e0bc8f420b2f393db0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } } }
I20260812 11:20:31.297302 8915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:31.297609 8914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "847cedabfec648e0bc8f420b2f393db0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847cedabfec648e0bc8f420b2f393db0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34303 } } }
I20260812 11:20:31.297695 8914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:31.301216 8942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:31.301294 8942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:31.301364 8932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:31.302206 8932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:31.304016 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:31.295136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:32829 (local address 127.0.0.1:34303)
0812 11:20:31.295256 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:31.295282 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:31.295501 (+ 219us) server_negotiation.cc:407] Connection header received
0812 11:20:31.295611 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:31.295614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:31.295666 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:31.295744 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:31.296534 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:31.297337 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:31.298421 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:31.298751 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:31.298852 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:31.300200 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:31.300203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:31.300206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:31.300230 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:31.301780 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:31.302367 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:31.302370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:31.302372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:31.302457 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:31.302876 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:31.302879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:31.302881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:31.303100 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:31.303237 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:31.303592 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 11:20:31.303840 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:20:31.307971 8932 catalog_manager.cc:1347] Generated new cluster ID: 7c6235be2b714c1daeec58dc6039db65
I20260812 11:20:31.308032 8932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:31.315804 8932 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:31.316560 8932 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:31.324090 8932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 847cedabfec648e0bc8f420b2f393db0: Generated new TSK 0
I20260812 11:20:31.324656 8932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:31.333316 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8744
W20260812 11:20:31.366492 8718 connection.cc:535] client connection to 127.0.0.1:34303 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34303): Connection reset by peer (error 104)
2026-08-12T11:20:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8951](info): set up 2 sockets
Aug 12 11:20:31 dist-test-slave-gjrz krb5kdc[8951](info): commencing operation
krb5kdc: starting...
W20260812 11:20:32.554297 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:32 dist-test-slave-gjrz krb5kdc[8951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:32.667961 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:32.774061 8977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:32.774307 8977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:32.774358 8977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:32.778282 8977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:32.778331 8977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:32.778348 8977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:32.778363 8977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:32.778376 8977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:32.783517 8977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:32.784425 8977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:32.785326 8977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:32.793298 8987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:32.793440 8977 server_base.cc:1034] running on GCE node
W20260812 11:20:32.793339 8997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:32.793612 8989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:32.794140 8977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:32.795126 8977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:32.796285 8977 hybrid_clock.cc:648] HybridClock initialized: now 1786533632796269 us; error 29 us; skew 500 ppm
Aug 12 11:20:32 dist-test-slave-gjrz krb5kdc[8951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:32.798864 8977 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:32.799700 8977 webserver.cc:460] Webserver started at http://127.0.0.1:32865/ using document root <none> and password file <none>
I20260812 11:20:32.800213 8977 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:32.800279 8977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:32.800468 8977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:32.802407 8977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "25c882979aff400b9cbd88c340850c21"
format_stamp: "Formatted at 2026-08-12 11:20:32 on dist-test-slave-gjrz"
server_key: "21de18f54a4a1f15aad4a66ed702c464"
server_key_iv: "0de36968f9c1f0944b9e5eb3d4550a71"
server_key_version: "encryptionkey@0"
I20260812 11:20:32.802965 8977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "25c882979aff400b9cbd88c340850c21"
format_stamp: "Formatted at 2026-08-12 11:20:32 on dist-test-slave-gjrz"
server_key: "21de18f54a4a1f15aad4a66ed702c464"
server_key_iv: "0de36968f9c1f0944b9e5eb3d4550a71"
server_key_version: "encryptionkey@0"
I20260812 11:20:32.806437 8977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:32.808861 9011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:32.809962 8977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:32.810119 8977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "25c882979aff400b9cbd88c340850c21"
format_stamp: "Formatted at 2026-08-12 11:20:32 on dist-test-slave-gjrz"
server_key: "21de18f54a4a1f15aad4a66ed702c464"
server_key_iv: "0de36968f9c1f0944b9e5eb3d4550a71"
server_key_version: "encryptionkey@0"
I20260812 11:20:32.810213 8977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:32.819363 8977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:32.822196 8977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:32.822357 8977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:32.828855 8977 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45891
I20260812 11:20:32.828899 9119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45891 every 8 connection(s)
I20260812 11:20:32.829728 8977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:32.832463 9121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:32.833014 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 8977
I20260812 11:20:32.833146 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:32.833426 31742 external_mini_cluster.cc:1428] Setting key 0bf432df6060353f80fe8c44fd28ee4e
I20260812 11:20:32.838833 9121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21: Bootstrap starting.
I20260812 11:20:32.841269 9121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:32.842134 9121 log.cc:824] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:32.842917 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:32.834714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56397 (local address 127.0.0.1:45891)
0812 11:20:32.835104 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:32.835123 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:32.835165 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:20:32.835999 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:32.836023 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:32.836430 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:32.836781 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:32.837167 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:32.838595 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:32.839506 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:32.840179 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:32.840338 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:32.841070 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:32.841076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:32.841090 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:32.841584 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:32.841646 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:32.841676 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:32.841804 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:32.841969 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 11:20:32.842097 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":96,"threads_started":1}
I20260812 11:20:32.844254 9121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21: No bootstrap required, opened a new log
I20260812 11:20:32.845676 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:32.847761 9121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } }
I20260812 11:20:32.847960 9121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:32.848021 9121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c882979aff400b9cbd88c340850c21, State: Initialized, Role: FOLLOWER
I20260812 11:20:32.848402 9121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } }
I20260812 11:20:32.848533 9121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:32.848596 9121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:32.848691 9121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:32.849463 9121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } }
I20260812 11:20:32.849763 9121 leader_election.cc:302] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c882979aff400b9cbd88c340850c21; no voters:
I20260812 11:20:32.850029 9121 leader_election.cc:288] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:32.850118 9144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:32.850313 9144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [term 1 LEADER]: Becoming Leader. State: Replica: 25c882979aff400b9cbd88c340850c21, State: Running, Role: LEADER
I20260812 11:20:32.850651 9144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } }
I20260812 11:20:32.851246 9121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:32.851475 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:32.847507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:46049 (local address 127.0.0.1:45891)
0812 11:20:32.847616 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:32.847620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:32.847800 (+ 180us) server_negotiation.cc:407] Connection header received
0812 11:20:32.847897 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:32.847900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:32.847943 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:32.848010 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:32.848361 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:32.848970 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:32.849771 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:32.850051 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:32.850143 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:32.851018 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:32.851022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:32.851025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:32.851036 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:32.851060 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:32.851078 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:32.851148 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:32.851320 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 11:20:32.851352 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:32.852459 9147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25c882979aff400b9cbd88c340850c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } } }
I20260812 11:20:32.852581 9147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:32.852806 9147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c882979aff400b9cbd88c340850c21. Latest consensus state: current_term: 1 leader_uuid: "25c882979aff400b9cbd88c340850c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c882979aff400b9cbd88c340850c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45891 } } }
I20260812 11:20:32.852882 9147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:32.852955 9166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:32.855912 9166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:32.861176 9166 catalog_manager.cc:1347] Generated new cluster ID: 6dcc4500b884474698b671fbfcdf8a90
I20260812 11:20:32.861238 9166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:32.873729 9166 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:32.874866 9166 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:32.891181 9166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25c882979aff400b9cbd88c340850c21: Generated new TSK 0
I20260812 11:20:32.891880 9166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:32.899705 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 8977
W20260812 11:20:32.912429 8948 connection.cc:535] client connection to 127.0.0.1:45891 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45891): Connection reset by peer (error 104)
2026-08-12T11:20:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:32 dist-test-slave-gjrz krb5kdc[9196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:32 dist-test-slave-gjrz krb5kdc[9196](info): set up 2 sockets
Aug 12 11:20:32 dist-test-slave-gjrz krb5kdc[9196](info): commencing operation
krb5kdc: starting...
W20260812 11:20:34.097550 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:34 dist-test-slave-gjrz krb5kdc[9196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:20:34.210872 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:34.318394 9230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:34.318620 9230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:34.318674 9230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:34.322556 9230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:34.322608 9230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:34.322636 9230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:34.322657 9230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:34.322686 9230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:34.327684 9230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37871
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:34.328578 9230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:34.329487 9230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:34.337477 9241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:34.337486 9245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:34.337534 9230 server_base.cc:1034] running on GCE node
W20260812 11:20:34.337486 9239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:34.338182 9230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:34.339157 9230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:34.340318 9230 hybrid_clock.cc:648] HybridClock initialized: now 1786533634340299 us; error 34 us; skew 500 ppm
Aug 12 11:20:34 dist-test-slave-gjrz krb5kdc[9196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:34.342819 9230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:20:34.343780 9230 webserver.cc:460] Webserver started at http://127.0.0.1:44853/ using document root <none> and password file <none>
I20260812 11:20:34.344318 9230 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:34.344384 9230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:34.344568 9230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:34.346462 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "65f03d2365284155b336e5f177440a9f"
format_stamp: "Formatted at 2026-08-12 11:20:34 on dist-test-slave-gjrz"
server_key: "89fb494cfacbac42ca8bbece464b13bb"
server_key_iv: "c0a7dca438d0789b58903f1dc3b595bb"
server_key_version: "encryptionkey@0"
I20260812 11:20:34.346971 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "65f03d2365284155b336e5f177440a9f"
format_stamp: "Formatted at 2026-08-12 11:20:34 on dist-test-slave-gjrz"
server_key: "89fb494cfacbac42ca8bbece464b13bb"
server_key_iv: "c0a7dca438d0789b58903f1dc3b595bb"
server_key_version: "encryptionkey@0"
I20260812 11:20:34.350481 9230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:34.352946 9256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:34.354111 9230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 11:20:34.354228 9230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "65f03d2365284155b336e5f177440a9f"
format_stamp: "Formatted at 2026-08-12 11:20:34 on dist-test-slave-gjrz"
server_key: "89fb494cfacbac42ca8bbece464b13bb"
server_key_iv: "c0a7dca438d0789b58903f1dc3b595bb"
server_key_version: "encryptionkey@0"
I20260812 11:20:34.354316 9230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:34.358814 9230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:34.361697 9230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:34.361864 9230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:34.368150 9230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37871
I20260812 11:20:34.368184 9384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37871 every 8 connection(s)
I20260812 11:20:34.368952 9230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:34.371719 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:34.375773 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9230
I20260812 11:20:34.375911 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:34.376163 31742 external_mini_cluster.cc:1428] Setting key a3d16366d0e18668e0a194e46c613991
I20260812 11:20:34.376869 9385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f: Bootstrap starting.
I20260812 11:20:34.379102 9385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:34.379827 9385 log.cc:824] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:34.380959 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:34.377396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:60961 (local address 127.0.0.1:37871)
0812 11:20:34.377797 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:34.377809 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:34.377842 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:34.378420 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:34.378435 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:34.378698 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:34.379001 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:34.379871 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:34.379875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:34.379886 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:34.379893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:34.379945 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:34.379971 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:34.380080 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:34.380186 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 11:20:34.380336 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":140,"threads_started":1}
I20260812 11:20:34.381765 9385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f: No bootstrap required, opened a new log
I20260812 11:20:34.383531 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:34.385502 9385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } }
I20260812 11:20:34.385754 9385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:34.385815 9385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f03d2365284155b336e5f177440a9f, State: Initialized, Role: FOLLOWER
I20260812 11:20:34.386188 9385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } }
I20260812 11:20:34.386312 9385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:34.386376 9385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:34.386469 9385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:34.387225 9385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } }
I20260812 11:20:34.387558 9385 leader_election.cc:302] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f03d2365284155b336e5f177440a9f; no voters:
I20260812 11:20:34.387828 9385 leader_election.cc:288] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:34.387908 9408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:34.388072 9408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [term 1 LEADER]: Becoming Leader. State: Replica: 65f03d2365284155b336e5f177440a9f, State: Running, Role: LEADER
I20260812 11:20:34.388346 9408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } }
I20260812 11:20:34.388767 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:34.385505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:36893 (local address 127.0.0.1:37871)
0812 11:20:34.385596 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:34.385600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:34.387532 (+ 1932us) server_negotiation.cc:407] Connection header received
0812 11:20:34.387632 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:34.387635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:34.387666 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:34.387730 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:34.388358 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:34.388362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:34.388364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:34.388376 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:34.388402 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:34.388415 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:34.388471 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:34.388626 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 11:20:34.388659 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 11:20:34.389009 9385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:34.390143 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f03d2365284155b336e5f177440a9f. Latest consensus state: current_term: 1 leader_uuid: "65f03d2365284155b336e5f177440a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } } }
I20260812 11:20:34.390272 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:34.390588 9416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f03d2365284155b336e5f177440a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03d2365284155b336e5f177440a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37871 } } }
I20260812 11:20:34.390646 9428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:34.390700 9416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:34.393482 9428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:34.398782 9428 catalog_manager.cc:1347] Generated new cluster ID: 16d60f4c9c7c4387b419fd8e2af4b1ea
I20260812 11:20:34.398836 9428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:34.413992 9428 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:34.414261 9428 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:34.419039 9428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f03d2365284155b336e5f177440a9f: Generated new TSK 0
I20260812 11:20:34.419767 9428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:34.422813 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9230
2026-08-12T11:20:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:34 dist-test-slave-gjrz krb5kdc[9455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:34 dist-test-slave-gjrz krb5kdc[9455](info): set up 2 sockets
Aug 12 11:20:34 dist-test-slave-gjrz krb5kdc[9455](info): commencing operation
krb5kdc: starting...
W20260812 11:20:35.620903 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:35 dist-test-slave-gjrz krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:35.736302 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:46185
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:39375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:46185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:35.842455 9484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:35.842675 9484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:35.842712 9484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:35.846498 9484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:35.846540 9484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:35.846557 9484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:35.846578 9484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:35.846591 9484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:35.851469 9484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:39375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:46185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:46185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:35.852286 9484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:35.853139 9484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:35.860886 9498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:35.860932 9493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:35.860895 9491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:35.861142 9484 server_base.cc:1034] running on GCE node
I20260812 11:20:35.861789 9484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:35.862774 9484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:35.863937 9484 hybrid_clock.cc:648] HybridClock initialized: now 1786533635863919 us; error 28 us; skew 500 ppm
Aug 12 11:20:35 dist-test-slave-gjrz krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533635, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:35.866389 9484 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:35.867182 9484 webserver.cc:460] Webserver started at http://127.30.255.190:39129/ using document root <none> and password file <none>
I20260812 11:20:35.867682 9484 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:35.867748 9484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:35.867920 9484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:35.869773 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "8ba99d33665747e49a0923e1fbda503c"
format_stamp: "Formatted at 2026-08-12 11:20:35 on dist-test-slave-gjrz"
server_key: "cc9c7771dcde9acbe05b9ba1f20906c2"
server_key_iv: "35b0582009277d0c1d7eb8d35652b683"
server_key_version: "encryptionkey@0"
I20260812 11:20:35.870293 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "8ba99d33665747e49a0923e1fbda503c"
format_stamp: "Formatted at 2026-08-12 11:20:35 on dist-test-slave-gjrz"
server_key: "cc9c7771dcde9acbe05b9ba1f20906c2"
server_key_iv: "35b0582009277d0c1d7eb8d35652b683"
server_key_version: "encryptionkey@0"
I20260812 11:20:35.873733 9484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:35.876019 9514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:35.877157 9484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:35.877266 9484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "8ba99d33665747e49a0923e1fbda503c"
format_stamp: "Formatted at 2026-08-12 11:20:35 on dist-test-slave-gjrz"
server_key: "cc9c7771dcde9acbe05b9ba1f20906c2"
server_key_iv: "35b0582009277d0c1d7eb8d35652b683"
server_key_version: "encryptionkey@0"
I20260812 11:20:35.877352 9484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:35.893508 9484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:35.895854 9484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:35.896018 9484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:35.902381 9484 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:46185
I20260812 11:20:35.902426 9612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:46185 every 8 connection(s)
I20260812 11:20:35.903196 9484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:35.905714 9613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:35.910665 9613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: Bootstrap starting.
I20260812 11:20:35.911353 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9484
I20260812 11:20:35.911499 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:35.911748 31742 external_mini_cluster.cc:1428] Setting key e6b65d5bf6f4b0e1ca71b18bd8232ce8
I20260812 11:20:35.913204 9613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:35.913877 9613 log.cc:824] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:35.915691 9613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: No bootstrap required, opened a new log
I20260812 11:20:35.918823 9613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } }
I20260812 11:20:35.919015 9613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:35.919071 9613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba99d33665747e49a0923e1fbda503c, State: Initialized, Role: FOLLOWER
I20260812 11:20:35.919476 9613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } }
Aug 12 11:20:35 dist-test-slave-gjrz krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:35.919610 9613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:35.919672 9613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:35.919770 9613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:35.920781 9613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } }
I20260812 11:20:35.921161 9613 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba99d33665747e49a0923e1fbda503c; no voters:
I20260812 11:20:35.921495 9613 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:35.921566 9624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:35.921783 9624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [term 1 LEADER]: Becoming Leader. State: Replica: 8ba99d33665747e49a0923e1fbda503c, State: Running, Role: LEADER
I20260812 11:20:35.922144 9624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } }
I20260812 11:20:35.922644 9613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:35.923715 9629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba99d33665747e49a0923e1fbda503c. Latest consensus state: current_term: 1 leader_uuid: "8ba99d33665747e49a0923e1fbda503c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } } }
I20260812 11:20:35.923856 9629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:35.924166 9628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ba99d33665747e49a0923e1fbda503c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba99d33665747e49a0923e1fbda503c" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 46185 } } }
I20260812 11:20:35.924258 9628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:35.926467 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:35.912696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50385 (local address 127.30.255.190:46185)
0812 11:20:35.913246 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:35.913262 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:35.913301 (+ 39us) server_negotiation.cc:407] Connection header received
0812 11:20:35.913892 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:35.913907 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:35.914246 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:35.914527 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:35.915338 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:35.916321 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:35.917160 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:35.917632 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:35.917774 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:35.920341 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:35.920349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:35.920366 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:35.920401 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:35.922474 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:35.923058 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:35.923065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:35.923071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:35.923225 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:35.923704 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:35.923707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:35.923709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:35.924249 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:35.925235 (+ 986us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:35.925396 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 11:20:35.925615 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":113,"threads_started":1}
W20260812 11:20:35.928092 9659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:35.928193 9659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:35.928272 9639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:35.929119 9639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:35.929378 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:35.934106 9639 catalog_manager.cc:1347] Generated new cluster ID: eda4c74330804a45a10aba15de6d2eea
I20260812 11:20:35.934172 9639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:35.940104 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:35.931573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60201 (local address 127.30.255.190:46185)
0812 11:20:35.931670 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:35.931674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:35.931903 (+ 229us) server_negotiation.cc:407] Connection header received
0812 11:20:35.932008 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:35.932011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:35.932063 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:35.932146 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:35.932859 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:35.933641 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:35.934780 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:35.935136 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:35.935232 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:35.936738 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:35.936741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:35.936744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:35.936767 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:35.938253 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:35.938752 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:35.938755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:35.938757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:35.938827 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:35.939130 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:35.939133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:35.939134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:35.939351 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:35.939476 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:35.939721 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 11:20:35.939802 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:20:35.955200 9639 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:35.955996 9639 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:35.962779 9639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba99d33665747e49a0923e1fbda503c: Generated new TSK 0
I20260812 11:20:35.963387 9639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:35.966784 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9484
W20260812 11:20:35.979290 9453 connection.cc:535] client connection to 127.30.255.190:46185 recv error: Network error: failed to read from TLS socket (remote: 127.30.255.190:46185): Connection reset by peer (error 104)
2026-08-12T11:20:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:36 dist-test-slave-gjrz krb5kdc[9681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:36 dist-test-slave-gjrz krb5kdc[9681](info): set up 2 sockets
Aug 12 11:20:36 dist-test-slave-gjrz krb5kdc[9681](info): commencing operation
krb5kdc: starting...
W20260812 11:20:37.176699 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:37 dist-test-slave-gjrz krb5kdc[9681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:37.289283 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36109
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:46325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:37.395596 9705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:37.395817 9705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:37.395871 9705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:37.399696 9705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:37.399745 9705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:37.399770 9705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:37.399792 9705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:37.399820 9705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:37.404798 9705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:46325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:37.405673 9705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:37.406576 9705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:37.414848 9718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:37.414968 9723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:37.414928 9705 server_base.cc:1034] running on GCE node
W20260812 11:20:37.414928 9719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:37.415653 9705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:37.416633 9705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:37.417784 9705 hybrid_clock.cc:648] HybridClock initialized: now 1786533637417766 us; error 29 us; skew 500 ppm
Aug 12 11:20:37 dist-test-slave-gjrz krb5kdc[9681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533637, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:37.434460 9705 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:37.435314 9705 webserver.cc:460] Webserver started at http://127.30.255.190:46687/ using document root <none> and password file <none>
I20260812 11:20:37.435827 9705 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:37.435885 9705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:37.436076 9705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:37.437985 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "8394f95738dc495d9aa4fce80109c99e"
format_stamp: "Formatted at 2026-08-12 11:20:37 on dist-test-slave-gjrz"
server_key: "9dc6314e5b5d888f3149009f2867e64b"
server_key_iv: "c3444ddb92da22da502a750292e23e82"
server_key_version: "encryptionkey@0"
I20260812 11:20:37.438524 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "8394f95738dc495d9aa4fce80109c99e"
format_stamp: "Formatted at 2026-08-12 11:20:37 on dist-test-slave-gjrz"
server_key: "9dc6314e5b5d888f3149009f2867e64b"
server_key_iv: "c3444ddb92da22da502a750292e23e82"
server_key_version: "encryptionkey@0"
I20260812 11:20:37.442065 9705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 11:20:37.444424 9733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:37.445537 9705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:20:37.445650 9705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "8394f95738dc495d9aa4fce80109c99e"
format_stamp: "Formatted at 2026-08-12 11:20:37 on dist-test-slave-gjrz"
server_key: "9dc6314e5b5d888f3149009f2867e64b"
server_key_iv: "c3444ddb92da22da502a750292e23e82"
server_key_version: "encryptionkey@0"
I20260812 11:20:37.445744 9705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:37.461558 9705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:37.464826 9705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:37.465000 9705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:37.471535 9705 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:36109
I20260812 11:20:37.471568 9834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:36109 every 8 connection(s)
I20260812 11:20:37.472417 9705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:37.474321 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9705
I20260812 11:20:37.474462 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:37.474731 31742 external_mini_cluster.cc:1428] Setting key b7ec1b647177a2a51b632ab5024dcc61
I20260812 11:20:37.475165 9837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:37.481894 9837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: Bootstrap starting.
I20260812 11:20:37.484213 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:37.476067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51661 (local address 127.30.255.190:36109)
0812 11:20:37.476488 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:37.476509 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:37.476551 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:20:37.477264 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:37.477287 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:37.477705 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:37.478054 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:37.478437 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:37.479898 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:37.480816 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:37.481486 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:37.481650 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:37.482387 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:37.482393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:37.482407 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:37.482863 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:37.482930 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:37.482962 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:37.483096 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:37.483187 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 11:20:37.483340 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260812 11:20:37.484447 9837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:37.485174 9837 log.cc:824] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:37.487021 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:37.487128 9837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: No bootstrap required, opened a new log
I20260812 11:20:37.491415 9837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } }
I20260812 11:20:37.491631 9837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:37.491724 9837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8394f95738dc495d9aa4fce80109c99e, State: Initialized, Role: FOLLOWER
I20260812 11:20:37.492163 9837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } }
I20260812 11:20:37.492365 9837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:37.492481 9837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:37.492638 9837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:37.493717 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:37.489179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39359 (local address 127.30.255.190:36109)
0812 11:20:37.489291 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:37.489296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:37.489407 (+ 111us) server_negotiation.cc:407] Connection header received
0812 11:20:37.489515 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:37.489518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:37.489563 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:37.489636 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:37.489851 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:37.490580 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:37.491788 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:37.492195 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:37.492295 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:37.493182 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:37.493186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:37.493189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:37.493203 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:37.493232 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:37.493246 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:37.493336 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:37.493543 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 11:20:37.493582 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:20:37.493718 9837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } }
I20260812 11:20:37.494242 9837 leader_election.cc:302] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8394f95738dc495d9aa4fce80109c99e; no voters:
I20260812 11:20:37.494606 9837 leader_election.cc:288] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:37.494982 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:37.495205 9861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [term 1 LEADER]: Becoming Leader. State: Replica: 8394f95738dc495d9aa4fce80109c99e, State: Running, Role: LEADER
I20260812 11:20:37.495604 9861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } }
I20260812 11:20:37.495890 9837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:37.497244 9862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8394f95738dc495d9aa4fce80109c99e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } } }
I20260812 11:20:37.497367 9862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:37.497740 9860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8394f95738dc495d9aa4fce80109c99e. Latest consensus state: current_term: 1 leader_uuid: "8394f95738dc495d9aa4fce80109c99e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8394f95738dc495d9aa4fce80109c99e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36109 } } }
I20260812 11:20:37.497885 9860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:37.500648 9884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:37.500752 9884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:37.500826 9880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:37.501705 9880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:37.506652 9880 catalog_manager.cc:1347] Generated new cluster ID: b72aa78272d84873b770532b9758ceab
I20260812 11:20:37.506712 9880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:37.515518 9880 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:37.516621 9880 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:37.522715 9880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8394f95738dc495d9aa4fce80109c99e: Generated new TSK 0
I20260812 11:20:37.523322 9880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:37.532755 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9705
W20260812 11:20:37.562198 9675 connection.cc:535] client connection to 127.30.255.190:36109 recv error: Network error: failed to read from TLS socket (remote: 127.30.255.190:36109): Connection reset by peer (error 104)
2026-08-12T11:20:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:37 dist-test-slave-gjrz krb5kdc[9894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:37 dist-test-slave-gjrz krb5kdc[9894](info): set up 2 sockets
Aug 12 11:20:37 dist-test-slave-gjrz krb5kdc[9894](info): commencing operation
krb5kdc: starting...
W20260812 11:20:38.750553 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:38 dist-test-slave-gjrz krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:38.863217 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:45367
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:37001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:45367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:38.968586 9922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:38.968802 9922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:38.968840 9922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:38.972721 9922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:38.972766 9922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:38.972784 9922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:38.972798 9922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:38.972817 9922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:38.977880 9922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:37001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:45367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:45367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:38.978731 9922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:38.979627 9922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:38.987255 9942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:38.987282 9945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:38.987303 9936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:38.987463 9922 server_base.cc:1034] running on GCE node
I20260812 11:20:38.988029 9922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:38.988976 9922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:38.990137 9922 hybrid_clock.cc:648] HybridClock initialized: now 1786533638990121 us; error 33 us; skew 500 ppm
Aug 12 11:20:38 dist-test-slave-gjrz krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533638, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:38.992653 9922 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:38.993367 9922 webserver.cc:460] Webserver started at http://127.30.255.190:44467/ using document root <none> and password file <none>
I20260812 11:20:38.993816 9922 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:38.993870 9922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:38.994030 9922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:38.995828 9922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "19ff4acd0c7a400f877089befb62ca6e"
format_stamp: "Formatted at 2026-08-12 11:20:38 on dist-test-slave-gjrz"
server_key: "062af5c9b19e0c03c3c346f284b2b285"
server_key_iv: "a82708015508df3ed2e31b0263d17a34"
server_key_version: "encryptionkey@0"
I20260812 11:20:38.996312 9922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "19ff4acd0c7a400f877089befb62ca6e"
format_stamp: "Formatted at 2026-08-12 11:20:38 on dist-test-slave-gjrz"
server_key: "062af5c9b19e0c03c3c346f284b2b285"
server_key_iv: "a82708015508df3ed2e31b0263d17a34"
server_key_version: "encryptionkey@0"
I20260812 11:20:38.999572 9922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:39.001729 9959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:39.002820 9922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:39.002921 9922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "19ff4acd0c7a400f877089befb62ca6e"
format_stamp: "Formatted at 2026-08-12 11:20:38 on dist-test-slave-gjrz"
server_key: "062af5c9b19e0c03c3c346f284b2b285"
server_key_iv: "a82708015508df3ed2e31b0263d17a34"
server_key_version: "encryptionkey@0"
I20260812 11:20:39.002997 9922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:39.007179 9922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:39.009876 9922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:39.010013 9922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:39.016352 9922 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:45367
I20260812 11:20:39.016430 10055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:45367 every 8 connection(s)
I20260812 11:20:39.017144 9922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:39.017966 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 9922
I20260812 11:20:39.018122 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:39.018419 31742 external_mini_cluster.cc:1428] Setting key 2c00dfe39bb42629e9e96cd8ae9898af
I20260812 11:20:39.020284 10056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:39.023103 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:39.019685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56765 (local address 127.30.255.190:45367)
0812 11:20:39.020118 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:39.020135 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:39.020168 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:39.020790 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:39.020809 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:39.021081 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:39.021339 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:39.022052 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:39.022056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:39.022067 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:39.022076 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:39.022122 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:39.022139 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:39.022240 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:39.022326 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 11:20:39.022467 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":160,"threads_started":1}
I20260812 11:20:39.025563 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:39.025990 10056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e: Bootstrap starting.
I20260812 11:20:39.028546 10056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:39.029271 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:39.027751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44179 (local address 127.30.255.190:45367)
0812 11:20:39.027863 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:39.027867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:39.028161 (+ 294us) server_negotiation.cc:407] Connection header received
0812 11:20:39.028202 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:39.028204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:39.028231 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:39.028297 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:39.028939 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:39.028942 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:39.028944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:39.028954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:39.028977 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:39.028986 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:39.029053 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:39.029148 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 11:20:39.029175 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:20:39.029407 10056 log.cc:824] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:39.031212 10056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e: No bootstrap required, opened a new log
I20260812 11:20:39.034651 10056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } }
I20260812 11:20:39.034853 10056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:39.034910 10056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ff4acd0c7a400f877089befb62ca6e, State: Initialized, Role: FOLLOWER
I20260812 11:20:39.035347 10056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } }
I20260812 11:20:39.035488 10056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:39.035553 10056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:39.035645 10056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:39.036453 10056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } }
I20260812 11:20:39.036758 10056 leader_election.cc:302] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19ff4acd0c7a400f877089befb62ca6e; no voters:
I20260812 11:20:39.037032 10056 leader_election.cc:288] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:39.037115 10084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:39.037314 10084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [term 1 LEADER]: Becoming Leader. State: Replica: 19ff4acd0c7a400f877089befb62ca6e, State: Running, Role: LEADER
I20260812 11:20:39.037670 10084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } }
I20260812 11:20:39.038183 10056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:39.039187 10088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19ff4acd0c7a400f877089befb62ca6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } } }
I20260812 11:20:39.039281 10090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19ff4acd0c7a400f877089befb62ca6e. Latest consensus state: current_term: 1 leader_uuid: "19ff4acd0c7a400f877089befb62ca6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ff4acd0c7a400f877089befb62ca6e" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45367 } } }
I20260812 11:20:39.039419 10088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:39.039485 10090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:39.039897 10105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:39.042613 10105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:39.047322 10105 catalog_manager.cc:1347] Generated new cluster ID: 6d39a822566142f29563f956dee95312
I20260812 11:20:39.047384 10105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:39.075548 10105 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:39.075835 10105 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:39.085613 10105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19ff4acd0c7a400f877089befb62ca6e: Generated new TSK 0
I20260812 11:20:39.086201 10105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:39.095072 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 9922
2026-08-12T11:20:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:39 dist-test-slave-gjrz krb5kdc[10125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:39 dist-test-slave-gjrz krb5kdc[10125](info): set up 2 sockets
Aug 12 11:20:39 dist-test-slave-gjrz krb5kdc[10125](info): commencing operation
krb5kdc: starting...
W20260812 11:20:40.296480 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:40.407455 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:45619
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:45619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:40.512687 10157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:40.512904 10157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:40.512943 10157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:40.516769 10157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:40.516817 10157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:40.516835 10157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:40.516861 10157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:40.516891 10157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:40.521831 10157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:45619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:45619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:40.522722 10157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:40.523649 10157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:40.531394 10167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:40.531418 10165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:40.531574 10157 server_base.cc:1034] running on GCE node
W20260812 11:20:40.531734 10169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:40.532238 10157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:40.533217 10157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:40.534365 10157 hybrid_clock.cc:648] HybridClock initialized: now 1786533640534347 us; error 36 us; skew 500 ppm
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533640, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:40.536811 10157 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:40.537626 10157 webserver.cc:460] Webserver started at http://127.30.255.190:45595/ using document root <none> and password file <none>
I20260812 11:20:40.538122 10157 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:40.538184 10157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:40.538365 10157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:40.540369 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "e72281be9a9f42fb9addf0adbcf65e0b"
format_stamp: "Formatted at 2026-08-12 11:20:40 on dist-test-slave-gjrz"
server_key: "743b493688bc20caa78890f334f34cc8"
server_key_iv: "ee6f43d23ff5152f055dd73a50caa811"
server_key_version: "encryptionkey@0"
I20260812 11:20:40.540891 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "e72281be9a9f42fb9addf0adbcf65e0b"
format_stamp: "Formatted at 2026-08-12 11:20:40 on dist-test-slave-gjrz"
server_key: "743b493688bc20caa78890f334f34cc8"
server_key_iv: "ee6f43d23ff5152f055dd73a50caa811"
server_key_version: "encryptionkey@0"
I20260812 11:20:40.544339 10157 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 11:20:40.546469 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:40.547644 10157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:40.547752 10157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "e72281be9a9f42fb9addf0adbcf65e0b"
format_stamp: "Formatted at 2026-08-12 11:20:40 on dist-test-slave-gjrz"
server_key: "743b493688bc20caa78890f334f34cc8"
server_key_iv: "ee6f43d23ff5152f055dd73a50caa811"
server_key_version: "encryptionkey@0"
I20260812 11:20:40.547838 10157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:40.559444 10157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:40.561837 10157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:40.562006 10157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:40.568480 10157 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:45619
I20260812 11:20:40.568496 10290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:45619 every 8 connection(s)
I20260812 11:20:40.569308 10157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:40.571895 10292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:40.572209 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10157
I20260812 11:20:40.572345 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:40.572602 31742 external_mini_cluster.cc:1428] Setting key 5e11631ca2960ae08da2bad91ed966e2
I20260812 11:20:40.578650 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: Bootstrap starting.
I20260812 11:20:40.581073 10292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: Neither blocks nor log segments found. Creating new log.
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:40.581866 10292 log.cc:824] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:40.584014 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: No bootstrap required, opened a new log
I20260812 11:20:40.586786 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:40.573840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39579 (local address 127.30.255.190:45619)
0812 11:20:40.574246 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:40.574265 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:40.574307 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:20:40.575006 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:40.575030 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:40.575478 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:40.575844 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:40.576480 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:40.577958 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:40.578760 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:40.579428 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:40.579593 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:40.582019 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:40.582026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:40.582042 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:40.582072 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:40.583954 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:40.584367 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:40.584373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:40.584378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:40.584514 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:40.584744 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:40.584747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:40.584749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:40.585217 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:40.585403 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:40.585579 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 11:20:40.585864 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260812 11:20:40.588253 10292 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } }
I20260812 11:20:40.588471 10292 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:40.588573 10292 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72281be9a9f42fb9addf0adbcf65e0b, State: Initialized, Role: FOLLOWER
I20260812 11:20:40.589017 10292 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } }
I20260812 11:20:40.589169 10292 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:40.589232 10292 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:40.589327 10292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:40.589463 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:40.590137 10292 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } }
I20260812 11:20:40.590440 10292 leader_election.cc:302] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72281be9a9f42fb9addf0adbcf65e0b; no voters:
I20260812 11:20:40.590725 10292 leader_election.cc:288] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:40.590812 10310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:40.590974 10310 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [term 1 LEADER]: Becoming Leader. State: Replica: e72281be9a9f42fb9addf0adbcf65e0b, State: Running, Role: LEADER
I20260812 11:20:40.591367 10310 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } }
I20260812 11:20:40.591948 10292 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:40.592598 10310 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72281be9a9f42fb9addf0adbcf65e0b. Latest consensus state: current_term: 1 leader_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } } }
I20260812 11:20:40.592733 10310 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:40.592953 10312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72281be9a9f42fb9addf0adbcf65e0b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 45619 } } }
I20260812 11:20:40.593091 10312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:40.596848 10341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:40.596951 10341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:40.597039 10329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:40.597697 10329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:40.599480 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:40.591397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49291 (local address 127.30.255.190:45619)
0812 11:20:40.591493 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:40.591497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:40.591683 (+ 186us) server_negotiation.cc:407] Connection header received
0812 11:20:40.591778 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:40.591781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:40.591824 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:40.591891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:40.592556 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:40.593293 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:40.594290 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:40.594604 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:40.594676 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:40.596142 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:40.596146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:40.596149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:40.596173 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:40.597755 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:40.598218 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:40.598221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:40.598223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:40.598310 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:40.598595 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:40.598598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:40.598600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:40.598805 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:40.598931 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:40.599186 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 11:20:40.599324 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:20:40.603229 10329 catalog_manager.cc:1347] Generated new cluster ID: e8b508e0320f4b7aa2e9647f85b2b297
I20260812 11:20:40.603309 10329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:40.620815 10329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:40.621927 10329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:40.625908 10329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e72281be9a9f42fb9addf0adbcf65e0b: Generated new TSK 0
I20260812 11:20:40.626469 10329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:40.634212 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10157
2026-08-12T11:20:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10352](info): set up 2 sockets
Aug 12 11:20:40 dist-test-slave-gjrz krb5kdc[10352](info): commencing operation
krb5kdc: starting...
W20260812 11:20:41.840732 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:41 dist-test-slave-gjrz krb5kdc[10352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:41.951697 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39153
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:41629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:39153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:42.056795 10375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:42.057019 10375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:42.057075 10375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:42.061297 10375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:42.061352 10375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:42.061378 10375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:42.061400 10375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:42.061429 10375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:42.066360 10375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:41629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:39153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:39153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:42.067222 10375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:42.068125 10375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:20:42.075947 10375 server_base.cc:1034] running on GCE node
W20260812 11:20:42.075914 10388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:42.075969 10385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:42.075914 10390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:42.076675 10375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:42.077656 10375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:42.078819 10375 hybrid_clock.cc:648] HybridClock initialized: now 1786533642078790 us; error 39 us; skew 500 ppm
Aug 12 11:20:42 dist-test-slave-gjrz krb5kdc[10352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533642, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:42.081387 10375 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:42.082177 10375 webserver.cc:460] Webserver started at http://127.30.255.190:44711/ using document root <none> and password file <none>
I20260812 11:20:42.082660 10375 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:42.082716 10375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:42.082887 10375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:42.084805 10375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "1ca24929030d4077ba2570e99cdbf6c3"
format_stamp: "Formatted at 2026-08-12 11:20:42 on dist-test-slave-gjrz"
server_key: "130e3d9f86b6bcaefa93dda778781263"
server_key_iv: "caf5c04eecd1f5dd0106a05d9c791b89"
server_key_version: "encryptionkey@0"
I20260812 11:20:42.085317 10375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "1ca24929030d4077ba2570e99cdbf6c3"
format_stamp: "Formatted at 2026-08-12 11:20:42 on dist-test-slave-gjrz"
server_key: "130e3d9f86b6bcaefa93dda778781263"
server_key_iv: "caf5c04eecd1f5dd0106a05d9c791b89"
server_key_version: "encryptionkey@0"
I20260812 11:20:42.089022 10375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:20:42.091185 10399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:42.092350 10375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:42.092480 10375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "1ca24929030d4077ba2570e99cdbf6c3"
format_stamp: "Formatted at 2026-08-12 11:20:42 on dist-test-slave-gjrz"
server_key: "130e3d9f86b6bcaefa93dda778781263"
server_key_iv: "caf5c04eecd1f5dd0106a05d9c791b89"
server_key_version: "encryptionkey@0"
I20260812 11:20:42.092574 10375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:42.102051 10375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:42.104842 10375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:42.105005 10375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:42.111487 10375 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:39153
I20260812 11:20:42.111524 10503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:39153 every 8 connection(s)
I20260812 11:20:42.112327 10375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:42.114868 10504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:42.116693 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10375
I20260812 11:20:42.116829 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:42.117093 31742 external_mini_cluster.cc:1428] Setting key 392417b5ac9c9684d0b9f78d52523849
I20260812 11:20:42.120766 10504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: Bootstrap starting.
I20260812 11:20:42.123157 10504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:42.124007 10504 log.cc:824] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:42.126593 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:42.118318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42861 (local address 127.30.255.190:39153)
0812 11:20:42.118734 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:42.118751 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:42.118792 (+ 41us) server_negotiation.cc:407] Connection header received
0812 11:20:42.119701 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:42.119724 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:42.120135 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:42.120465 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:42.120841 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:42.122297 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:42.123217 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:42.123890 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:42.124052 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:42.124939 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:42.124945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:42.124959 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:42.125353 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:42.125408 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:42.125432 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:42.125512 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:42.125641 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 11:20:42.125812 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
I20260812 11:20:42.127655 10504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: No bootstrap required, opened a new log
I20260812 11:20:42.129273 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:42.131335 10504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } }
I20260812 11:20:42.131604 10504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:42.131664 10504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca24929030d4077ba2570e99cdbf6c3, State: Initialized, Role: FOLLOWER
I20260812 11:20:42.132057 10504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } }
I20260812 11:20:42.132181 10504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:42.132238 10504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:42.132329 10504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:42.133091 10504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } }
I20260812 11:20:42.133391 10504 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca24929030d4077ba2570e99cdbf6c3; no voters:
I20260812 11:20:42.133663 10504 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:42.133741 10522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:42.133916 10522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca24929030d4077ba2570e99cdbf6c3, State: Running, Role: LEADER
I20260812 11:20:42.134192 10522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } }
I20260812 11:20:42.134829 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:42.131097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43163 (local address 127.30.255.190:39153)
0812 11:20:42.131197 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:42.131201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:42.131326 (+ 125us) server_negotiation.cc:407] Connection header received
0812 11:20:42.131419 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:42.131422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:42.131464 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:42.131540 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:42.131809 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:42.132379 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:42.133143 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:42.133415 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:42.133506 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:42.134239 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:42.134243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:42.134245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:42.134258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:42.134284 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:42.134294 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:42.134353 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:42.134697 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 11:20:42.134739 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:20:42.134835 10504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:42.135614 10526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca24929030d4077ba2570e99cdbf6c3. Latest consensus state: current_term: 1 leader_uuid: "1ca24929030d4077ba2570e99cdbf6c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } } }
I20260812 11:20:42.135735 10526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:42.136049 10525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca24929030d4077ba2570e99cdbf6c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca24929030d4077ba2570e99cdbf6c3" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 39153 } } }
I20260812 11:20:42.136142 10525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3 [sys.catalog]: This master's current role is: LEADER
W20260812 11:20:42.139088 10551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:42.139150 10551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:42.139222 10542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:42.139899 10542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:42.144495 10542 catalog_manager.cc:1347] Generated new cluster ID: 252708573503430190eb9294599c8df5
I20260812 11:20:42.144553 10542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:42.150624 10542 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:42.151772 10542 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:42.156864 10542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca24929030d4077ba2570e99cdbf6c3: Generated new TSK 0
I20260812 11:20:42.157539 10542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:42.164652 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10375
2026-08-12T11:20:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:42 dist-test-slave-gjrz krb5kdc[10562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:42 dist-test-slave-gjrz krb5kdc[10562](info): set up 2 sockets
Aug 12 11:20:42 dist-test-slave-gjrz krb5kdc[10562](info): commencing operation
krb5kdc: starting...
W20260812 11:20:43.365079 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:43 dist-test-slave-gjrz krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:43.475419 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43799
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:43.583402 10590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:43.583616 10590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:43.583654 10590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:43.587468 10590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:43.587515 10590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:43.587533 10590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:43.587558 10590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:43.587585 10590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:43.592479 10590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:43.593325 10590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:43.594197 10590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:20:43.601976 10590 server_base.cc:1034] running on GCE node
W20260812 11:20:43.602000 10603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:43.602046 10599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:43.601994 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:43.602653 10590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:43.603632 10590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:43.604775 10590 hybrid_clock.cc:648] HybridClock initialized: now 1786533643604755 us; error 34 us; skew 500 ppm
Aug 12 11:20:43 dist-test-slave-gjrz krb5kdc[10562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:43.610376 10590 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:43.611178 10590 webserver.cc:460] Webserver started at http://127.30.255.190:40769/ using document root <none> and password file <none>
I20260812 11:20:43.611687 10590 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:43.611752 10590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:43.611922 10590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:43.613787 10590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "20f50540ff9b4aa1afacd14cf6240e33"
format_stamp: "Formatted at 2026-08-12 11:20:43 on dist-test-slave-gjrz"
server_key: "4109b8e58c71051c5abe475ad1985710"
server_key_iv: "23400c03531bc89f9527d1d1974028af"
server_key_version: "encryptionkey@0"
I20260812 11:20:43.614291 10590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "20f50540ff9b4aa1afacd14cf6240e33"
format_stamp: "Formatted at 2026-08-12 11:20:43 on dist-test-slave-gjrz"
server_key: "4109b8e58c71051c5abe475ad1985710"
server_key_iv: "23400c03531bc89f9527d1d1974028af"
server_key_version: "encryptionkey@0"
I20260812 11:20:43.617671 10590 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:20:43.619851 10625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:43.620968 10590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:43.621076 10590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "20f50540ff9b4aa1afacd14cf6240e33"
format_stamp: "Formatted at 2026-08-12 11:20:43 on dist-test-slave-gjrz"
server_key: "4109b8e58c71051c5abe475ad1985710"
server_key_iv: "23400c03531bc89f9527d1d1974028af"
server_key_version: "encryptionkey@0"
I20260812 11:20:43.621166 10590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:43.625334 10590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:43.627847 10590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:43.627996 10590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:43.634294 10590 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:43799
I20260812 11:20:43.634311 10731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:43799 every 8 connection(s)
I20260812 11:20:43.635078 10590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:43.637601 10736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:43.640517 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10590
I20260812 11:20:43.640666 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:43.640966 31742 external_mini_cluster.cc:1428] Setting key 6b2392cfa65b2f3670946d70fbb27d3a
I20260812 11:20:43.643090 10736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33: Bootstrap starting.
I20260812 11:20:43.645372 10736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:43.645581 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:43.642165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39777 (local address 127.30.255.190:43799)
0812 11:20:43.642528 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:43.642543 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:43.642577 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:20:43.643179 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:43.643203 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:43.643497 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:43.643754 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:43.644483 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:43.644488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:43.644499 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:43.644509 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:43.644562 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:43.644581 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:43.644681 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:43.644772 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 11:20:43.644900 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260812 11:20:43.646106 10736 log.cc:824] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:43.648046 10736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33: No bootstrap required, opened a new log
I20260812 11:20:43.648144 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:43.651439 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:43.649982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36321 (local address 127.30.255.190:43799)
0812 11:20:43.650090 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:43.650094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:43.650328 (+ 234us) server_negotiation.cc:407] Connection header received
0812 11:20:43.650431 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:43.650433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:43.650459 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:43.650516 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:43.651111 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:43.651114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:43.651115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:20:43.651125 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:43.651146 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:20:43.651155 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:43.651217 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:43.651312 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 11:20:43.651344 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:20:43.651873 10736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } }
I20260812 11:20:43.652124 10736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:20:43.652225 10736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f50540ff9b4aa1afacd14cf6240e33, State: Initialized, Role: FOLLOWER
I20260812 11:20:43.652640 10736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } }
I20260812 11:20:43.652760 10736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:20:43.652804 10736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:20:43.652872 10736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:43.653607 10736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } }
I20260812 11:20:43.653880 10736 leader_election.cc:302] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20f50540ff9b4aa1afacd14cf6240e33; no voters:
I20260812 11:20:43.654111 10736 leader_election.cc:288] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:20:43.654196 10759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:43.654392 10759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [term 1 LEADER]: Becoming Leader. State: Replica: 20f50540ff9b4aa1afacd14cf6240e33, State: Running, Role: LEADER
I20260812 11:20:43.654709 10759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } }
I20260812 11:20:43.655143 10736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:43.656163 10760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20f50540ff9b4aa1afacd14cf6240e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } } }
I20260812 11:20:43.656292 10760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:43.656590 10761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20f50540ff9b4aa1afacd14cf6240e33. Latest consensus state: current_term: 1 leader_uuid: "20f50540ff9b4aa1afacd14cf6240e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f50540ff9b4aa1afacd14cf6240e33" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 43799 } } }
I20260812 11:20:43.656654 10770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:43.656675 10761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:43.659578 10770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:43.665777 10770 catalog_manager.cc:1347] Generated new cluster ID: ad01d3243b074508bd6bec2d19c372a4
I20260812 11:20:43.665839 10770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:43.687671 10770 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:43.687958 10770 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:43.697765 10770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20f50540ff9b4aa1afacd14cf6240e33: Generated new TSK 0
I20260812 11:20:43.698335 10770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:20:43.705618 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10590
2026-08-12T11:20:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1528 ms)
[----------] 18 tests from AuthTokenIssuingTest (27836 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:43 dist-test-slave-gjrz krb5kdc[10796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:43 dist-test-slave-gjrz krb5kdc[10796](info): set up 2 sockets
Aug 12 11:20:43 dist-test-slave-gjrz krb5kdc[10796](info): commencing operation
krb5kdc: starting...
W20260812 11:20:44.896265 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:44 dist-test-slave-gjrz krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:45.008359 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36553
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:45.114988 10829 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:45.115221 10829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:45.115242 10829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:45.115303 10829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:45.119192 10829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:45.119239 10829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:45.119257 10829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:45.119294 10829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:45.119319 10829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:45.124256 10829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:36553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:45.125089 10829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:45.125934 10829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:45.133885 10842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:45.133908 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:45.133878 10843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:45.134235 10829 server_base.cc:1034] running on GCE node
I20260812 11:20:45.134739 10829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:45.135782 10829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:45.136941 10829 hybrid_clock.cc:648] HybridClock initialized: now 1786533645136931 us; error 27 us; skew 500 ppm
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:45.139550 10829 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:45.140352 10829 webserver.cc:460] Webserver started at http://127.30.255.190:32785/ using document root <none> and password file <none>
I20260812 11:20:45.140851 10829 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:45.140916 10829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:45.141139 10829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:45.143133 10829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "f5daba486a0a4832ab3fcda38f17499b"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "f20f991508852bbe7d224d2d67840b05"
server_key_iv: "e72a3af5adf666c2b7009427587a412f"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.143676 10829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "f5daba486a0a4832ab3fcda38f17499b"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "f20f991508852bbe7d224d2d67840b05"
server_key_iv: "e72a3af5adf666c2b7009427587a412f"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.147116 10829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:45.149391 10864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:45.150532 10829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:45.150642 10829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "f5daba486a0a4832ab3fcda38f17499b"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "f20f991508852bbe7d224d2d67840b05"
server_key_iv: "e72a3af5adf666c2b7009427587a412f"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.150728 10829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:45.183044 10829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:45.186164 10829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:45.186347 10829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:45.193230 10829 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:36553
I20260812 11:20:45.193511 10955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:36553 every 8 connection(s)
I20260812 11:20:45.194247 10829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:45.197007 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:45.200839 10958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.203758 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10829
I20260812 11:20:45.203900 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:45.204198 31742 external_mini_cluster.cc:1428] Setting key d825b33f22af0194570867074dae212f
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.255.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.189@KRBTEST.COM" created.
I20260812 11:20:45.214838 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.202974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36553 (local address 127.0.0.1:45554)
0812 11:20:45.203442 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.203486 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.204394 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.205033 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.205038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.205469 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.205716 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.205729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.206842 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.206845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.207617 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.207627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.208219 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.208223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.208334 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.208826 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.208842 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.210414 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.212135 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.212143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.212153 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.212370 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.212663 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.212666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.212668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.212772 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.213302 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.213307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.213433 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.213577 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.213933 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":85,"thread_start_us":112,"threads_started":1}
I20260812 11:20:45.214838 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.203810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.30.255.190:36553)
0812 11:20:45.204096 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.204108 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.204154 (+ 46us) server_negotiation.cc:407] Connection header received
0812 11:20:45.204681 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.204685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.204858 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.205032 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.205858 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.206723 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.207722 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.208080 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.208223 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.210551 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.210556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.210566 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.210587 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.212035 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.212468 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.212473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.212474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.212579 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.212853 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.212857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.212858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.213195 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.213308 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.213661 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.213848 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
I20260812 11:20:45.217305 10958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.217964 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.217731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53252)
0812 11:20:45.217838 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.217906 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 11:20:45.218183 10869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
I20260812 11:20:45.218932 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.218742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53266)
0812 11:20:45.218834 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.218879 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 11:20:45.219055 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:36865: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
I20260812 11:20:45.227372 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:36865
--webserver_interface=127.30.255.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:20:45.268272 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } attempt: 1
I20260812 11:20:45.268977 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.268719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53268)
0812 11:20:45.268849 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.268904 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:20:45.269661 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.269473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53280)
0812 11:20:45.269565 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.269607 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 11:20:45.269798 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:36865: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
W20260812 11:20:45.337106 10971 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:45.337332 10971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:45.337352 10971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:45.337385 10971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:20:45.337944 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } attempt: 2
I20260812 11:20:45.338692 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.338418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53290)
0812 11:20:45.338558 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.338617 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 11:20:45.339411 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.339160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53306)
0812 11:20:45.339289 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.339341 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 11:20:45.339556 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:36865: Network error: Client connection negotiation failed: client connection to 127.30.255.189:36865: connect: Connection refused (error 111)
W20260812 11:20:45.341490 10971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:45.341537 10971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:45.341564 10971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:45.341593 10971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:45.341622 10971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:45.346675 10971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:36865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.255.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:45.347623 10971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:45.348579 10971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:45.356489 10979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:45.356498 10980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:45.356642 10971 server_base.cc:1034] running on GCE node
W20260812 11:20:45.356503 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:45.357226 10971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:45.358199 10971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:45.359320 10971 hybrid_clock.cc:648] HybridClock initialized: now 1786533645359302 us; error 28 us; skew 500 ppm
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:45.361750 10971 init.cc:377] Logged in from keytab as kudu/127.30.255.189@KRBTEST.COM (short username kudu)
I20260812 11:20:45.362591 10971 webserver.cc:460] Webserver started at http://127.30.255.189:42987/ using document root <none> and password file <none>
I20260812 11:20:45.363085 10971 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:45.363148 10971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:45.363375 10971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:45.365235 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data/instance:
uuid: "78b0f1cc80874990b097cab425374ef5"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "99ee5c3ccf770ed4a1375c5830c3efde"
server_key_iv: "fb1e870f6fe7a817b6c334539025b60d"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.365741 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal/instance:
uuid: "78b0f1cc80874990b097cab425374ef5"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "99ee5c3ccf770ed4a1375c5830c3efde"
server_key_iv: "fb1e870f6fe7a817b6c334539025b60d"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.369135 10971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:45.371282 11002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:45.372377 10971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:45.372486 10971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "78b0f1cc80874990b097cab425374ef5"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "99ee5c3ccf770ed4a1375c5830c3efde"
server_key_iv: "fb1e870f6fe7a817b6c334539025b60d"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.372571 10971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:45.381682 10971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:45.384070 10971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:45.384220 10971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:45.390223 10971 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.189:36865
I20260812 11:20:45.390456 11106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.189:36865 every 8 connection(s)
I20260812 11:20:45.391034 10971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
I20260812 11:20:45.393013 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 10971
I20260812 11:20:45.393157 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-1/wal/instance
I20260812 11:20:45.393448 31742 external_mini_cluster.cc:1428] Setting key b3c47616e55d24fe8b1d76721ae9c5f4
I20260812 11:20:45.393796 11109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:45.397516 11109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.255.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.188@KRBTEST.COM" created.
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:45.410239 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.399247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.30.255.190:36553)
0812 11:20:45.399367 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.399370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.400318 (+ 948us) server_negotiation.cc:407] Connection header received
0812 11:20:45.401169 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.401173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.401231 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.401336 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.402431 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.402974 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.404107 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.404345 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.404436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.406803 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.406808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.406810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.406843 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.408150 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.408750 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.408753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.408755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.408833 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.409191 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.409193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.409194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.409355 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.409425 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.410023 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.410122 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:45.411255 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.399734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36553 (local address 127.0.0.1:45570)
0812 11:20:45.400185 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.400223 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.400969 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.401500 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.401508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.401808 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.402286 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.402300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.403097 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.403099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.403977 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.403987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.404480 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.404484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.404653 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.405193 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.405213 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.406638 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.408268 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.408274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.408289 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.408639 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.408919 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.408923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.408925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.409109 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.409458 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.409464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.409765 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.409994 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.410304 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":129,"threads_started":1}
I20260812 11:20:45.412602 11109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
I20260812 11:20:45.419445 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:33579
--webserver_interface=127.30.255.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:36241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:20:45.420563 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.413047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53314)
0812 11:20:45.413146 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.413157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.413214 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.413736 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.413739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.413880 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.413949 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.413954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.414843 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.414846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.415378 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.415387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.415907 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.415911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.415991 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.416325 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.416335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.417446 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.419166 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.419169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.419170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.419408 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.419698 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.419700 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.419701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.419776 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.420230 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.420232 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.420304 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.420376 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.420461 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 11:20:45.420795 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.413159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.30.255.189:36865)
0812 11:20:45.413428 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.413437 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.413453 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:20:45.413484 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.413487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.413630 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.413736 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.414046 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.414740 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.415475 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.415809 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.415898 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.417590 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.417601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.417613 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.417640 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.419059 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.419510 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.419517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.419518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.419614 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.419858 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.419861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.419863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.420140 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.420270 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.420484 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.420625 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
I20260812 11:20:45.422480 11109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.423111 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.422833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57884)
0812 11:20:45.422958 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.423048 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 11:20:45.423339 11012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.424127 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.423899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57898)
0812 11:20:45.424014 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.424069 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 11:20:45.424276 11109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.430697 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } attempt: 3
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:20:45.439157 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.431053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53330)
0812 11:20:45.431176 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.431190 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.431294 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.431504 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.431507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.431682 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.431789 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.431795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.432465 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.432468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.433802 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.433812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.434655 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.434659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.434769 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.435151 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.435164 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.436404 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.437837 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.437840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.437842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.438035 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.438281 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.438284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.438286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.438363 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.438695 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.438698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.438788 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.438873 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.439007 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 11:20:45.439155 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.431121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.30.255.189:36865)
0812 11:20:45.431210 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.431213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.431226 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:45.431377 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.431379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.431414 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.431474 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.431896 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.432380 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.433911 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.434543 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.434622 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.436534 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.436537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.436539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.436559 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.437736 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.438124 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.438127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.438128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.438198 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.438450 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.438453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.438455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.438605 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.438688 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.438932 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.439026 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:20:45.439868 10958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.440449 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.440237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57908)
0812 11:20:45.440335 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.440384 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 11:20:45.441035 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.440817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57912)
0812 11:20:45.440938 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.440985 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 11:20:45.441138 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.472292 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 1
I20260812 11:20:45.472990 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.472745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57920)
0812 11:20:45.472870 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.472924 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:20:45.473464 11109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 1
I20260812 11:20:45.473660 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.473440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57926)
0812 11:20:45.473548 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.473592 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 11:20:45.473790 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.474107 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.473859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57938)
0812 11:20:45.473980 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.474037 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 11:20:45.474715 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.474512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57948)
0812 11:20:45.474613 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.474661 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 11:20:45.474830 11109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
W20260812 11:20:45.530893 11124 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:45.531153 11124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:45.531188 11124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:45.531244 11124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:45.535128 11124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:45.535176 11124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:45.535195 11124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:45.535224 11124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:45.535249 11124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:45.540268 11124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:36241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:36553,127.30.255.189:36865,127.30.255.188:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.255.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:45.541155 11124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:45.542089 11124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:20:45.542984 11109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 2
I20260812 11:20:45.543777 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.543472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57962)
0812 11:20:45.543619 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.543688 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 11:20:45.544471 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.544248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57970)
0812 11:20:45.544352 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.544406 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 11:20:45.544595 11109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.548924 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 2
I20260812 11:20:45.549582 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.549314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57976)
0812 11:20:45.549445 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.549499 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 11:20:45.550107 11134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:45.550200 11138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:45.550268 11136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:45.550396 11124 server_base.cc:1034] running on GCE node
I20260812 11:20:45.550583 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.550256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:57982)
0812 11:20:45.550459 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.550511 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260812 11:20:45.550726 10958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:33579: Network error: Client connection negotiation failed: client connection to 127.30.255.188:33579: connect: Connection refused (error 111)
I20260812 11:20:45.551178 11124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:45.552161 11124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:45.553325 11124 hybrid_clock.cc:648] HybridClock initialized: now 1786533645553303 us; error 30 us; skew 500 ppm
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:45.555889 11124 init.cc:377] Logged in from keytab as kudu/127.30.255.188@KRBTEST.COM (short username kudu)
I20260812 11:20:45.556643 11124 webserver.cc:460] Webserver started at http://127.30.255.188:42277/ using document root <none> and password file <none>
I20260812 11:20:45.557145 11124 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:45.557209 11124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:45.557391 11124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:45.559231 11124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data/instance:
uuid: "3c9e5265a185442db32272debf10439e"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "abb4a153c8550bd44f662143f2922bf9"
server_key_iv: "e3daa8a822312301fb9bdbe12d6982a2"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.559772 11124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal/instance:
uuid: "3c9e5265a185442db32272debf10439e"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "abb4a153c8550bd44f662143f2922bf9"
server_key_iv: "e3daa8a822312301fb9bdbe12d6982a2"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.563117 11124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:20:45.565313 11146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:45.566392 11124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:45.566499 11124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "3c9e5265a185442db32272debf10439e"
format_stamp: "Formatted at 2026-08-12 11:20:45 on dist-test-slave-gjrz"
server_key: "abb4a153c8550bd44f662143f2922bf9"
server_key_iv: "e3daa8a822312301fb9bdbe12d6982a2"
server_key_version: "encryptionkey@0"
I20260812 11:20:45.566583 11124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:45.586120 11124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:45.588639 11124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:45.588791 11124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:45.594878 11124 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.188:33579
I20260812 11:20:45.595098 11242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.188:33579 every 8 connection(s)
I20260812 11:20:45.595741 11124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
I20260812 11:20:45.598186 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:45.601574 11243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.605130 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11124
I20260812 11:20:45.605247 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786533365564394-31742-0/minicluster-data/master-2/wal/instance
I20260812 11:20:45.605511 31742 external_mini_cluster.cc:1428] Setting key 819e8b79e27f21fe654c0b69d8b801d3
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:20:45.614807 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.603188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.30.255.190:36553)
0812 11:20:45.603337 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.603341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.604059 (+ 718us) server_negotiation.cc:407] Connection header received
0812 11:20:45.604885 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.604888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.604965 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.605057 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.606330 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.606986 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.608082 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.608296 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.608387 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.610685 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.610688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.610690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.610714 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.612746 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.613301 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.613305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.613307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.613400 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.613710 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.613712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.613713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.613903 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.614011 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.614530 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.614652 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:20:45.615563 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.603530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:36553 (local address 127.0.0.1:45574)
0812 11:20:45.603921 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.603959 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.604688 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.605256 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.605265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.605627 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.606152 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.606168 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.607088 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.607091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.607962 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.607972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.608427 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.608431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.608594 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.609095 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.609110 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.610552 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.612854 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.612864 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.612874 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.613199 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.613485 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.613487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.613488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.613616 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.614011 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.614016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.614277 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.614447 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.614688 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260812 11:20:45.615564 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.606094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.30.255.188:33579)
0812 11:20:45.606451 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.606458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.606487 (+ 29us) server_negotiation.cc:407] Connection header received
0812 11:20:45.606604 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.606608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.606742 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.606858 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.607638 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.608332 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.609295 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.609644 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.609749 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.612083 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.612090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.612102 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.612128 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.613552 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.613949 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.613956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.613957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.614044 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.614302 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.614304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.614305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.614545 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.614756 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.614958 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.615076 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":124,"threads_started":1}
I20260812 11:20:45.616730 11243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:45.624532 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.617163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:36865 (local address 127.0.0.1:53346)
0812 11:20:45.617247 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.617261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.617328 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.617797 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.617800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.617963 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.618081 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.618088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.618959 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.618962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.619716 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.619726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.620148 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.620151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.620222 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.620573 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.620587 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.621696 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.623168 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.623171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.623173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.623422 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.623683 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.623685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.623686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.623750 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.624112 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.624115 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.624206 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.624318 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.624428 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 11:20:45.624620 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.617427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.30.255.189:36865)
0812 11:20:45.617575 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.617579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.617593 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:20:45.617635 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.617638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.617684 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.617762 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.618192 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.618843 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.619822 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.620044 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.620141 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.621829 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.621833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.621835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.621861 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.623083 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.623514 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.623518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.623519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.623601 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.623821 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.623824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.623826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.624022 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.624107 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.624357 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.624463 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:20:45.625324 11243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:45.625662 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.617571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.30.255.190:36553)
0812 11:20:45.617660 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.617663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.617674 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:20:45.617737 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.617740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.617777 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.617842 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.618443 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.619086 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.620045 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.620309 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.620394 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.622513 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.622517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.622520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.622545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.623952 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.624386 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.624389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.624391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.624472 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.624767 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.624770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.624772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.624968 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.625170 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.625310 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.625411 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:20:45.633126 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.625712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:58004)
0812 11:20:45.625792 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.625804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.625876 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.626065 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.626067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.626178 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.626240 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.626244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.627058 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.627062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.627743 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.627753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.628313 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.628317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.628401 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.628764 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.628777 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.630005 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.631704 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.631707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.631709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.631915 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.632201 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.632204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.632206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.632297 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.632647 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.632650 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.632738 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.632849 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.632966 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 11:20:45.633214 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.625772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.30.255.188:33579)
0812 11:20:45.625869 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.625873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.625885 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:45.625944 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.625947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.625986 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.626059 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.626323 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.626953 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.627849 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.628184 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.628277 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.630143 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.630147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.630149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.630175 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.631602 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.632019 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.632022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.632024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.632107 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.632380 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.632383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.632385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.632560 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.632633 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.632881 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.633039 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":41}
I20260812 11:20:45.634793 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.626610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.30.255.189:36865)
0812 11:20:45.626706 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.626709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.626722 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:45.626770 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.626773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.626807 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.626875 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.627476 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.628056 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.629062 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.629259 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.629333 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.631618 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.631621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.631623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.631644 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.633043 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.633756 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.633760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.633761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.633848 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.634146 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.634148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.634149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.634290 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.634442 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.634607 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.634687 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:45.635722 11109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 3
I20260812 11:20:45.636675 11243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: Bootstrap starting.
I20260812 11:20:45.637910 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:20:45.639513 11243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:45.639034 10915 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f5daba486a0a4832ab3fcda38f17499b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45588
I20260812 11:20:45.640286 11243 log.cc:824] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:20:45.642863 11243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: No bootstrap required, opened a new log
I20260812 11:20:45.643512 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.636029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:58010)
0812 11:20:45.636175 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.636188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.636263 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.636450 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.636452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.636620 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.636703 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.636709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.637538 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.637542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.638290 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.638302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.638838 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.638842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.638922 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.639356 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.639371 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.640702 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.642144 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.642148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.642150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.642328 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.642576 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.642579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.642581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.642658 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.643054 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.643057 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.643141 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.643224 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.643365 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 11:20:45.643512 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.636092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.30.255.188:33579)
0812 11:20:45.636175 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.636179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.636229 (+ 50us) server_negotiation.cc:407] Connection header received
0812 11:20:45.636326 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.636329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.636364 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.636427 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.636826 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.637432 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.638416 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.638722 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.638825 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.640834 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.640837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.640857 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.640878 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.642045 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.642411 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.642415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.642416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.642490 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.642734 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.642737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.642739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.642910 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.643036 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.643305 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.643395 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:20:45.645856 11109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: Bootstrap starting.
I20260812 11:20:45.646843 11243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.647060 11243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c9e5265a185442db32272debf10439e, State: Initialized, Role: FOLLOWER
I20260812 11:20:45.647414 11243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.647904 11109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:45.647943 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.648171 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.648603 11109 log.cc:824] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:45.648718 11243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:45.649888 10958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } attempt: 3
I20260812 11:20:45.650393 11109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: No bootstrap required, opened a new log
W20260812 11:20:45.653802 11271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:45.653872 11271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:45.654628 11109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.654816 11109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b0f1cc80874990b097cab425374ef5, State: Initialized, Role: FOLLOWER
Aug 12 11:20:45 dist-test-slave-gjrz krb5kdc[10796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:20:45.655094 11109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.656255 11109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:45.652465 11124 master_runner.cc:186] Error getting master registration for 127.30.255.190:36553: OK, instance_id { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" instance_seqno: 1786533645191315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:20:45.656471 11124 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:45.658370 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.659155 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.650225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:33579 (local address 127.0.0.1:58022)
0812 11:20:45.650421 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:45.650437 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:45.650516 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:45.650840 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:45.650843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:45.651015 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:45.651117 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.651125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.652015 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.652019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.652788 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:45.652800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.653425 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.653429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:45.653517 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.653975 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:45.653989 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:45.655467 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:45.657362 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.657366 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.657368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.657615 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.657949 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:45.657952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:45.657954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:45.658107 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:45.658661 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:45.658664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:45.658775 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 11:20:45.658876 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 11:20:45.659013 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 11:20:45.659768 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.650296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.30.255.188:33579)
0812 11:20:45.650651 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.650655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.650668 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:20:45.650697 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.650700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.650736 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.650799 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.651248 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.651925 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.652920 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.653230 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.653317 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.655664 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.655668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.655670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.655694 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.657226 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.657737 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.657741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.657742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.657839 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.658225 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.658229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.658230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.658439 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.658527 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.659532 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.659637 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
W20260812 11:20:45.661314 11294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:45.661384 11294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:45.659123 10971 master_runner.cc:186] Error getting master registration for 127.30.255.190:36553: OK, instance_id { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" instance_seqno: 1786533645191315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:20:45.662177 10971 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:45.658484 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.662955 10958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: Bootstrap starting.
I20260812 11:20:45.665398 10958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:45.666231 10958 log.cc:824] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:45.668465 10958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: No bootstrap required, opened a new log
I20260812 11:20:45.671424 10958 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.671622 10958 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5daba486a0a4832ab3fcda38f17499b, State: Initialized, Role: FOLLOWER
I20260812 11:20:45.671897 10958 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.673035 10958 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:45.673928 10915 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 11:20:45.674052 10915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:45.675241 10915 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.676133 10915 leader_election.cc:288] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [CANDIDATE]: Term 1 election: Requested vote from peers 78b0f1cc80874990b097cab425374ef5 (127.30.255.189:36865), 3c9e5265a185442db32272debf10439e (127.30.255.188:33579)
I20260812 11:20:45.677038 11196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5daba486a0a4832ab3fcda38f17499b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c9e5265a185442db32272debf10439e"
I20260812 11:20:45.677278 11196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:45.678490 11196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5daba486a0a4832ab3fcda38f17499b in term 1.
I20260812 11:20:45.678442 11061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5daba486a0a4832ab3fcda38f17499b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "78b0f1cc80874990b097cab425374ef5"
I20260812 11:20:45.678642 11061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:45.678953 10869 leader_election.cc:302] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c9e5265a185442db32272debf10439e, f5daba486a0a4832ab3fcda38f17499b; no voters:
I20260812 11:20:45.679402 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.679579 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.679697 11304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:45.679800 11304 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [term 1 LEADER]: Becoming Leader. State: Replica: f5daba486a0a4832ab3fcda38f17499b, State: Running, Role: LEADER
I20260812 11:20:45.679890 11061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5daba486a0a4832ab3fcda38f17499b in term 1.
I20260812 11:20:45.680174 11304 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } }
I20260812 11:20:45.681905 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5daba486a0a4832ab3fcda38f17499b. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.682070 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:45.682446 11346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:45.684268 10829 master_runner.cc:263] No leader master found from master f5daba486a0a4832ab3fcda38f17499b
I20260812 11:20:45.684335 10829 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:45.685431 11346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 11:20:45.688135 10873 tablet.cc:2369] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:45.702494 11196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [term 1 FOLLOWER]: Refusing update from remote peer f5daba486a0a4832ab3fcda38f17499b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:45.703228 11344 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:45.706920 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5daba486a0a4832ab3fcda38f17499b. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.707036 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.708030 11061 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [term 1 FOLLOWER]: Refusing update from remote peer f5daba486a0a4832ab3fcda38f17499b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:45.708621 11344 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [LEADER]: Connected to new peer: Peer: permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:45.709942 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.688794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.30.255.189:36865)
0812 11:20:45.688899 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.688903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.691539 (+ 2636us) server_negotiation.cc:407] Connection header received
0812 11:20:45.691717 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.691720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.691765 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.691812 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.694077 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.694855 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.697952 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.698299 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.698368 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.702135 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.702139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.702142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.702168 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.703752 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.706591 (+ 2839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.706595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.706597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.706686 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.707977 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.707981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.707983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.708212 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.708334 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.709672 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.709789 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:45.711114 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.687638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.30.255.190:36553)
0812 11:20:45.687758 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.687762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.692923 (+ 5161us) server_negotiation.cc:407] Connection header received
0812 11:20:45.693093 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.693096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.693138 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.693182 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.696086 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.696820 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.699132 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.699501 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.699576 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.704276 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.704280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.704283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.704308 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.705873 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.708641 (+ 2768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.708645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.708646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.708732 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.709028 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.709031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.709033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.709233 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.709345 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.710825 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.710939 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:20:45.711815 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5daba486a0a4832ab3fcda38f17499b. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.711918 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.720377 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.720481 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:45.720575 11349 mvcc.cc:205] Tried to move back new op lower bound from 7317641812790296576 to 7317641812709351424. Current Snapshot: MvccSnapshot[applied={T|T < 7317641812790296576}]
I20260812 11:20:45.720754 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:45.689462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.30.255.188:33579)
0812 11:20:45.689561 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.689565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.706516 (+ 16951us) server_negotiation.cc:407] Connection header received
0812 11:20:45.707083 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.707086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.707132 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.707184 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.713454 (+ 6270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.714216 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.715391 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:45.715727 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:45.715797 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:45.717400 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:45.717404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:45.717406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:45.717431 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:45.718948 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.719470 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.719474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.719476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.719560 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:45.719850 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:45.719854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:45.719856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:45.720064 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:45.720178 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:45.720505 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 11:20:45.720613 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 11:20:45.720841 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.720942 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.721056 11353 mvcc.cc:205] Tried to move back new op lower bound from 7317641812790296576 to 7317641812709351424. Current Snapshot: MvccSnapshot[applied={T|T < 7317641812790296576}]
I20260812 11:20:45.723564 11346 catalog_manager.cc:1347] Generated new cluster ID: a872b241d4304e1ea7b2b83f7808913c
I20260812 11:20:45.723626 11346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:45.724400 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.724500 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:45.725531 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5daba486a0a4832ab3fcda38f17499b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5daba486a0a4832ab3fcda38f17499b" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 36553 } } peers { permanent_uuid: "78b0f1cc80874990b097cab425374ef5" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 36865 } } peers { permanent_uuid: "3c9e5265a185442db32272debf10439e" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 33579 } } }
I20260812 11:20:45.725647 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:45.726145 11357 mvcc.cc:205] Tried to move back new op lower bound from 7317641812790296576 to 7317641812709351424. Current Snapshot: MvccSnapshot[applied={T|T < 7317641812790296576}]
I20260812 11:20:45.751565 11346 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:45.752794 11346 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:45.764822 11346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5daba486a0a4832ab3fcda38f17499b: Generated new TSK 0
I20260812 11:20:45.765602 11346 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:20:45.799875 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.796850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.30.255.190:36553)
0812 11:20:45.796962 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.796966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.798769 (+ 1803us) server_negotiation.cc:407] Connection header received
0812 11:20:45.798960 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.798963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.799012 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.799082 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.799754 (+ 672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45596: BlockingRecv error: recv got EOF from 127.0.0.1:45596 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 11:20:45.801909 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.797581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.30.255.189:36865)
0812 11:20:45.797688 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.797692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.800513 (+ 2821us) server_negotiation.cc:407] Connection header received
0812 11:20:45.800964 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.800967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.801015 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.801076 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.801788 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53380: BlockingRecv error: recv got EOF from 127.0.0.1:53380 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 11:20:45.802779 11247 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.798181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.30.255.188:33579)
0812 11:20:45.798287 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.798291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.800721 (+ 2430us) server_negotiation.cc:407] Connection header received
0812 11:20:45.801206 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.801209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.801254 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.801298 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.802675 (+ 1377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58034: BlockingRecv error: recv got EOF from 127.0.0.1:58034 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 11:20:45.807071 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.805698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53384 (local address 127.30.255.189:36865)
0812 11:20:45.805794 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.805798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.806213 (+ 415us) server_negotiation.cc:407] Connection header received
0812 11:20:45.806406 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.806409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.806473 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.806521 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.806975 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53384: BlockingRecv error: recv got EOF from 127.0.0.1:53384 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 11:20:45.812186 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.810640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.30.255.189:36865)
0812 11:20:45.810738 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.810742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.811256 (+ 514us) server_negotiation.cc:407] Connection header received
0812 11:20:45.811502 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.811506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.811553 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.811602 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.812104 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53390: BlockingRecv error: recv got EOF from 127.0.0.1:53390 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 11:20:45.817798 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.816333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.30.255.189:36865)
0812 11:20:45.816427 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.816431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.816946 (+ 515us) server_negotiation.cc:407] Connection header received
0812 11:20:45.817151 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.817153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.817200 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.817248 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.817719 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53400: BlockingRecv error: recv got EOF from 127.0.0.1:53400 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 11:20:45.825232 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.823724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.30.255.189:36865)
0812 11:20:45.823845 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.823849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.824380 (+ 531us) server_negotiation.cc:407] Connection header received
0812 11:20:45.824562 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.824565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.824610 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.824659 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.825152 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53404: BlockingRecv error: recv got EOF from 127.0.0.1:53404 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 11:20:45.836611 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.835116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.30.255.189:36865)
0812 11:20:45.835205 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.835209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.835743 (+ 534us) server_negotiation.cc:407] Connection header received
0812 11:20:45.835906 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.835909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.835951 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.835995 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.836524 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53410: BlockingRecv error: recv got EOF from 127.0.0.1:53410 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 11:20:45.856155 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.854776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.30.255.189:36865)
0812 11:20:45.854879 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.854883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.855131 (+ 248us) server_negotiation.cc:407] Connection header received
0812 11:20:45.855245 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.855248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.855333 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.855422 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.856041 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53414: BlockingRecv error: recv got EOF from 127.0.0.1:53414 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 11:20:45.890655 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.889767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.30.255.189:36865)
0812 11:20:45.889906 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.889910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.889998 (+ 88us) server_negotiation.cc:407] Connection header received
0812 11:20:45.890114 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.890116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.890157 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.890222 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.890555 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53416: BlockingRecv error: recv got EOF from 127.0.0.1:53416 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 11:20:45.957545 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:45.956532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.30.255.189:36865)
0812 11:20:45.956666 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:45.956671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:45.956798 (+ 127us) server_negotiation.cc:407] Connection header received
0812 11:20:45.956932 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:45.956935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:45.956985 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:45.957055 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:45.957418 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53420: BlockingRecv error: recv got EOF from 127.0.0.1:53420 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 11:20:46.088585 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:46.087513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.30.255.189:36865)
0812 11:20:46.087652 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.087656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.087806 (+ 150us) server_negotiation.cc:407] Connection header received
0812 11:20:46.087940 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.087943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.087990 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.088070 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:46.088462 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53424: BlockingRecv error: recv got EOF from 127.0.0.1:53424 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 11:20:46.347675 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:46.346672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.30.255.189:36865)
0812 11:20:46.346789 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.346793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.346878 (+ 85us) server_negotiation.cc:407] Connection header received
0812 11:20:46.347007 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.347010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.347059 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.347137 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:46.347572 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53428: BlockingRecv error: recv got EOF from 127.0.0.1:53428 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:20:46.655249 11271 catalog_manager.cc:1259] Loaded cluster ID: a872b241d4304e1ea7b2b83f7808913c
I20260812 11:20:46.655339 11271 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: loading cluster ID for follower catalog manager: success
I20260812 11:20:46.657150 11271 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: acquiring CA information for follower catalog manager: success
I20260812 11:20:46.658293 11271 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c9e5265a185442db32272debf10439e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:20:46.662390 11294 catalog_manager.cc:1259] Loaded cluster ID: a872b241d4304e1ea7b2b83f7808913c
I20260812 11:20:46.662453 11294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: loading cluster ID for follower catalog manager: success
I20260812 11:20:46.663930 11294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: acquiring CA information for follower catalog manager: success
I20260812 11:20:46.664896 11294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 78b0f1cc80874990b097cab425374ef5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:20:46.865221 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:46.861775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.30.255.189:36865)
0812 11:20:46.862022 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.862026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.862042 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:20:46.862176 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.862178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.862306 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.862407 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:46.862623 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.863160 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.863963 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.864177 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.864246 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:46.864491 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:46.864915 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:46.865023 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:46.865052 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 11:20:46.865098 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":83,"threads_started":1}
I20260812 11:20:46.871145 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:46.868003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.30.255.188:33579)
0812 11:20:46.868212 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.868216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.868228 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:46.868325 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.868327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.868426 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.868521 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:46.868680 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.869160 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.869942 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.870158 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.870235 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:46.870470 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:46.870831 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:46.870914 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:46.870963 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 11:20:46.871007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260812 11:20:46.877382 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:46.873876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.30.255.189:36865)
0812 11:20:46.873992 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.873996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.874008 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:46.874132 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.874135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.874184 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.874229 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:46.874603 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.875162 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.876310 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.876584 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.876676 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:46.876970 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:46.877041 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:46.877077 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:46.877216 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:20:46.877262 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:20:46.878036 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:46.873700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.30.255.190:36553)
0812 11:20:46.874147 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.874151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.874163 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:46.874241 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.874244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.874437 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.874581 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:46.874776 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.875473 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.876479 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.876809 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.876894 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:46.877201 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:46.877461 (+ 260us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:46.877796 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:46.877830 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 11:20:46.877887 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":63,"threads_started":1}
I20260812 11:20:46.878242 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:46.874063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.30.255.188:33579)
0812 11:20:46.874163 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:46.874167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:46.874521 (+ 354us) server_negotiation.cc:407] Connection header received
0812 11:20:46.874676 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:46.874679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:46.874717 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:46.874762 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:46.875126 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.875874 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.876991 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:46.877279 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:46.877367 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:46.877695 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:46.877768 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:46.877811 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:46.878079 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 11:20:46.878129 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:20:46.880805 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10829
I20260812 11:20:46.896911 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 10971
I20260812 11:20:46.914057 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11124
2026-08-12T11:20:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3206 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:20:46 dist-test-slave-gjrz krb5kdc[11522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:20:46 dist-test-slave-gjrz krb5kdc[11522](info): set up 2 sockets
Aug 12 11:20:46 dist-test-slave-gjrz krb5kdc[11522](info): commencing operation
krb5kdc: starting...
W20260812 11:20:48.112556 31742 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/test-user.keytab.
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:20:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:20:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.255.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
Entry for principal kudu/127.30.255.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab.
I20260812 11:20:48.225461 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41737
--webserver_interface=127.30.255.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:20:48.330211 11546 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:48.330458 11546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:48.330490 11546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:48.330549 11546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:48.334389 11546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:48.334437 11546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:48.334462 11546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:48.334482 11546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:48.334507 11546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:48.339533 11546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.190.keytab
--principal=kudu/127.30.255.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.190:41737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.255.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:48.340409 11546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:48.341284 11546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:48.349080 11563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:48.349157 11559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:48.349080 11561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:48.349201 11546 server_base.cc:1034] running on GCE node
I20260812 11:20:48.349833 11546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:48.350764 11546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:48.351923 11546 hybrid_clock.cc:648] HybridClock initialized: now 1786533648351895 us; error 31 us; skew 500 ppm
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:48.354461 11546 init.cc:377] Logged in from keytab as kudu/127.30.255.190@KRBTEST.COM (short username kudu)
I20260812 11:20:48.355291 11546 webserver.cc:460] Webserver started at http://127.30.255.190:43577/ using document root <none> and password file <none>
I20260812 11:20:48.355783 11546 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:48.355849 11546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:48.356056 11546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:48.357887 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data/instance:
uuid: "0d9258cba5244f218489944454778546"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "583c0659e200750fb4866e484006edbd"
server_key_iv: "49c12b24b12daefe03e2a415090e1bc8"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.358405 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance:
uuid: "0d9258cba5244f218489944454778546"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "583c0659e200750fb4866e484006edbd"
server_key_iv: "49c12b24b12daefe03e2a415090e1bc8"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.361804 11546 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 11:20:48.364123 11580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:48.365222 11546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:20:48.365332 11546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal
uuid: "0d9258cba5244f218489944454778546"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "583c0659e200750fb4866e484006edbd"
server_key_iv: "49c12b24b12daefe03e2a415090e1bc8"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.365419 11546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:48.394498 11546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:48.397024 11546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:48.397183 11546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:48.403538 11546 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.190:41737
I20260812 11:20:48.403828 11680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.190:41737 every 8 connection(s)
I20260812 11:20:48.404464 11546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/data/info.pb
I20260812 11:20:48.406991 11683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:48.410660 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11546
I20260812 11:20:48.410770 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-0/wal/instance
I20260812 11:20:48.410480 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.411098 31742 external_mini_cluster.cc:1428] Setting key 72162c73c82a5f259eac44626a2cc797
WARNING: no policy specified for kudu/127.30.255.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.189@KRBTEST.COM" created.
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:48.428192 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.412917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41737 (local address 127.0.0.1:48896)
0812 11:20:48.413496 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.413541 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.414346 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.415029 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.415034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.415468 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.415811 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.415837 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.417034 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.417038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.418165 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.418176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.418782 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.418786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.418895 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.419434 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.419455 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.423661 (+ 4206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.425556 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.425566 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.425575 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.425847 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.426197 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.426199 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.426200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.426301 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.426985 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.426990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.427102 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.427227 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.427500 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"spinlock_wait_cycles":52992,"thread_start_us":137,"threads_started":1}
I20260812 11:20:48.428202 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.412917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.30.255.190:41737)
0812 11:20:48.413361 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.413379 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.413643 (+ 264us) server_negotiation.cc:407] Connection header received
0812 11:20:48.414641 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.414646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.414826 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.414998 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.416007 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.416898 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.418272 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.418630 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.418740 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.423815 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.423821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.423833 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.423859 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.425444 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.425971 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.425978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.425979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.426113 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.426385 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.426389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.426390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.426884 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.427021 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.427361 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.427614 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260812 11:20:48.430298 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.430923 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.430661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43274)
0812 11:20:48.430774 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.430856 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 11:20:48.431142 11596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
I20260812 11:20:48.431919 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.431741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43284)
0812 11:20:48.431831 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.431873 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 11:20:48.432060 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34633: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
Entry for principal kudu/127.30.255.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab.
I20260812 11:20:48.437777 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:34633
--webserver_interface=127.30.255.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:20:48.481285 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } attempt: 1
I20260812 11:20:48.482053 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.481776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43292)
0812 11:20:48.481922 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.481982 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 11:20:48.482803 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.482580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43298)
0812 11:20:48.482682 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.482736 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 11:20:48.482963 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34633: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
W20260812 11:20:48.543905 11693 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:48.544178 11693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:48.544222 11693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:48.544274 11693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:48.548156 11693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:48.548202 11693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:48.548228 11693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:48.548259 11693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:48.548288 11693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:48.551120 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } attempt: 2
I20260812 11:20:48.551887 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.551619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43312)
0812 11:20:48.551753 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.551813 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 11:20:48.552567 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.552354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43322)
0812 11:20:48.552462 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.552508 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 11:20:48.552675 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.189:34633: Network error: Client connection negotiation failed: client connection to 127.30.255.189:34633: connect: Connection refused (error 111)
I20260812 11:20:48.553485 11693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.189.keytab
--principal=kudu/127.30.255.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.189:34633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.255.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:48.554378 11693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:48.555296 11693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:20:48.563078 11706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:48.563145 11693 server_base.cc:1034] running on GCE node
W20260812 11:20:48.563107 11714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:48.563107 11708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:48.563746 11693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:48.564679 11693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:48.565828 11693 hybrid_clock.cc:648] HybridClock initialized: now 1786533648565807 us; error 29 us; skew 500 ppm
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:48.568329 11693 init.cc:377] Logged in from keytab as kudu/127.30.255.189@KRBTEST.COM (short username kudu)
I20260812 11:20:48.569139 11693 webserver.cc:460] Webserver started at http://127.30.255.189:41177/ using document root <none> and password file <none>
I20260812 11:20:48.569617 11693 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:48.569681 11693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:48.569913 11693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:48.571808 11693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data/instance:
uuid: "8bce4f7a724e47be812b90ced7722cff"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "8f34ba53b8a48c9e7d0f11843f328ec4"
server_key_iv: "01fc1eeabe3b61f80901b05d753e33a5"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.572314 11693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal/instance:
uuid: "8bce4f7a724e47be812b90ced7722cff"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "8f34ba53b8a48c9e7d0f11843f328ec4"
server_key_iv: "01fc1eeabe3b61f80901b05d753e33a5"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.575614 11693 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:20:48.577750 11728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:48.578802 11693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:20:48.578912 11693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal
uuid: "8bce4f7a724e47be812b90ced7722cff"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "8f34ba53b8a48c9e7d0f11843f328ec4"
server_key_iv: "01fc1eeabe3b61f80901b05d753e33a5"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.578994 11693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:48.588581 11693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:48.590934 11693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:48.591091 11693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:48.597186 11693 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.189:34633
I20260812 11:20:48.597424 11832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.189:34633 every 8 connection(s)
I20260812 11:20:48.597993 11693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/data/info.pb
I20260812 11:20:48.600535 11833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:48.603089 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11693
I20260812 11:20:48.603219 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-1/wal/instance
I20260812 11:20:48.603549 31742 external_mini_cluster.cc:1428] Setting key a51e9079928ea6b457253bae1518a4ee
I20260812 11:20:48.604269 11833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.255.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.255.188@KRBTEST.COM" created.
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:48.622781 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.606212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.30.255.190:41737)
0812 11:20:48.606350 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.606355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.607127 (+ 772us) server_negotiation.cc:407] Connection header received
0812 11:20:48.608038 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.608042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.608101 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.608189 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.609283 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.610099 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.611153 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.611547 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.611670 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.619420 (+ 7750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.619424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.619427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.619456 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.620922 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.621409 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.621414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.621416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.621506 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.621821 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.621825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.621826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.622065 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.622145 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.622568 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.622662 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:20:48.623631 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.606520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41737 (local address 127.0.0.1:48908)
0812 11:20:48.606967 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.607017 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.607825 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.608355 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.608362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.608675 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.609128 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.609141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.610206 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.610209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.611026 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.611035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.611690 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.611693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.611836 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.617738 (+ 5902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.617754 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.619229 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.621055 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.621060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.621069 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.621310 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.621593 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.621596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.621597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.621721 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.622162 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.622167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.622371 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.622532 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.622772 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
I20260812 11:20:48.624989 11833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
Entry for principal kudu/127.30.255.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab.
I20260812 11:20:48.628244 31742 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
/tmp/dist-test-taskoRXcNR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:45343
--webserver_interface=127.30.255.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.255.148:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:20:48.633479 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.625385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43326)
0812 11:20:48.625496 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.625509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.625578 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.626125 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.626128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.626279 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.626356 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.626363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.627359 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.627362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.628050 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.628061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.628664 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.628668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.628777 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.629150 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.629164 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.630275 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.632187 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.632189 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.632191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.632364 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.632637 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.632639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.632641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.632705 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.633151 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.633153 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.633223 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.633292 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.633377 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":17}
I20260812 11:20:48.633705 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.625494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.30.255.189:34633)
0812 11:20:48.625752 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.625760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.625776 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:20:48.625805 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.625809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.626016 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.626146 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.626448 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.627236 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.628154 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.628567 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.628655 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.630407 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.630428 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.630440 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.630465 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.632086 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.632463 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.632469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.632471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.632560 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.632778 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.632781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.632783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.633063 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.633197 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.633397 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.633545 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":185,"thread_start_us":113,"threads_started":1}
I20260812 11:20:48.635515 11833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.636116 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.635872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37462)
0812 11:20:48.635991 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.636066 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 11:20:48.636296 11738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
I20260812 11:20:48.637058 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.636871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37466)
0812 11:20:48.636967 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.637009 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 11:20:48.637195 11833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
I20260812 11:20:48.643814 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } attempt: 3
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:20:48.651957 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.644229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43342 (local address 127.30.255.189:34633)
0812 11:20:48.644332 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.644335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.644346 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:20:48.644437 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.644439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.644470 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.644535 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.645309 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.645768 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.646810 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.647454 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.647538 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.649381 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.649384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.649385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.649406 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.650585 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.650987 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.650989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.650990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.651052 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.651305 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.651307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.651308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.651453 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.651530 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.651778 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.651866 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:20:48.651983 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.644166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43342)
0812 11:20:48.644273 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.644287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.644359 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.644554 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.644557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.644725 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.645219 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.645227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.645855 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.645859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.646720 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.646730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.647562 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.647566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.647673 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.648024 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.648037 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.649256 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.650682 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.650686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.650688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.650909 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.651119 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.651122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.651124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.651206 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.651529 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.651532 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.651623 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.651723 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.651836 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 11:20:48.652781 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.653354 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.653169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37468)
0812 11:20:48.653261 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.653303 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 11:20:48.653930 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.653740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37470)
0812 11:20:48.653847 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.653885 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 11:20:48.654033 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
I20260812 11:20:48.685163 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 1
I20260812 11:20:48.685927 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.685654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37474)
0812 11:20:48.685790 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.685848 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 11:20:48.686403 11833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 1
I20260812 11:20:48.686614 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.686377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37478)
0812 11:20:48.686492 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.686547 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 11:20:48.686995 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
I20260812 11:20:48.687063 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.686813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37484)
0812 11:20:48.686932 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.686985 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 11:20:48.687727 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.687534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37486)
0812 11:20:48.687628 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.687672 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 11:20:48.687839 11833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
W20260812 11:20:48.740419 11845 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:20:48.740661 11845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:20:48.740702 11845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:20:48.740762 11845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:20:48.744634 11845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:20:48.744683 11845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:20:48.744707 11845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:20:48.744741 11845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:20:48.744771 11845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:20:48.749810 11845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.255.148:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.255.190:41737,127.30.255.189:34633,127.30.255.188:45343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/krb5kdc/kudu_127.30.255.188.keytab
--principal=kudu/127.30.255.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.255.188:45343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.255.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 09:58:34 UTC on ccycloud.quasar-qfmjiv.root.comops.site
build id 4582284
I20260812 11:20:48.750770 11845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:20:48.751748 11845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:20:48.755968 11833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 2
I20260812 11:20:48.756759 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.756492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37490)
0812 11:20:48.756628 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.756688 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 11:20:48.757437 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.757237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37492)
0812 11:20:48.757338 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.757383 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 11:20:48.757546 11833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
W20260812 11:20:48.759835 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:20:48.759871 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:48.760057 11845 server_base.cc:1034] running on GCE node
W20260812 11:20:48.760208 11855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:20:48.760684 11845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:20:48.761701 11845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:20:48.762159 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 2
I20260812 11:20:48.762777 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.762552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37506)
0812 11:20:48.762662 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.762712 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 11:20:48.762873 11845 hybrid_clock.cc:648] HybridClock initialized: now 1786533648762845 us; error 34 us; skew 500 ppm
I20260812 11:20:48.763398 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.763195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37510)
0812 11:20:48.763308 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.763348 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 11:20:48.763507 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.255.188:45343: Network error: Client connection negotiation failed: client connection to 127.30.255.188:45343: connect: Connection refused (error 111)
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:20:48.765434 11845 init.cc:377] Logged in from keytab as kudu/127.30.255.188@KRBTEST.COM (short username kudu)
I20260812 11:20:48.766264 11845 webserver.cc:460] Webserver started at http://127.30.255.188:37165/ using document root <none> and password file <none>
I20260812 11:20:48.766767 11845 fs_manager.cc:359] Metadata directory not provided
I20260812 11:20:48.766832 11845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:20:48.767063 11845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:20:48.769026 11845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data/instance:
uuid: "dac70f12fe704d86a69a512aca21bc5c"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "31bb744a0c7d862988747e79da8ca295"
server_key_iv: "1ef702e98ba97ee5eefcbf232435090b"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.769537 11845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal/instance:
uuid: "dac70f12fe704d86a69a512aca21bc5c"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "31bb744a0c7d862988747e79da8ca295"
server_key_iv: "1ef702e98ba97ee5eefcbf232435090b"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.772935 11845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:20:48.775128 11876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:20:48.776227 11845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:20:48.776342 11845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data,/tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal
uuid: "dac70f12fe704d86a69a512aca21bc5c"
format_stamp: "Formatted at 2026-08-12 11:20:48 on dist-test-slave-gjrz"
server_key: "31bb744a0c7d862988747e79da8ca295"
server_key_iv: "1ef702e98ba97ee5eefcbf232435090b"
server_key_version: "encryptionkey@0"
I20260812 11:20:48.776428 11845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:20:48.818815 11845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:20:48.821282 11845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:20:48.821440 11845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:20:48.827705 11845 rpc_server.cc:307] RPC server started. Bound to: 127.30.255.188:45343
I20260812 11:20:48.827934 11968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.255.188:45343 every 8 connection(s)
I20260812 11:20:48.828559 11845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/data/info.pb
I20260812 11:20:48.831051 11969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:20:48.833971 31742 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu as pid 11845
I20260812 11:20:48.834087 31742 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoRXcNR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786533365564394-31742-0/minicluster-data/master-2/wal/instance
I20260812 11:20:48.834333 31742 external_mini_cluster.cc:1428] Setting key 1b915e602657ac03a25e5453f0a688bf
I20260812 11:20:48.834576 11969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:20:48.845790 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.836301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.30.255.190:41737)
0812 11:20:48.836410 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.836415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.836712 (+ 297us) server_negotiation.cc:407] Connection header received
0812 11:20:48.837038 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.837041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.837098 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.837166 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.837915 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.838407 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.839351 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.839573 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.839662 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.842267 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.842271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.842274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.842303 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.843772 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.844262 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.844265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.844267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.844358 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.844700 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.844703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.844704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.844931 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.845002 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.845538 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.845639 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 11:20:48.846515 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.835530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.30.255.188:45343)
0812 11:20:48.835889 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.835904 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.835944 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:20:48.836815 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.836837 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.837065 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.837222 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.838027 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.838831 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.839713 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.840142 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.840248 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.842651 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.842658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.842671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.842695 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.844273 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.844708 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.844714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.844715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.844787 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.845123 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.845126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.845127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.845345 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.845498 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.845738 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.845860 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
I20260812 11:20:48.846516 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.836238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.190:41737 (local address 127.0.0.1:48922)
0812 11:20:48.836593 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.836628 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.836872 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.837229 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.837234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.837546 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.837777 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.837791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.838504 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.838507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.839226 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.839235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.839704 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.839708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.839895 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.840422 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.840443 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.842086 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.843871 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.843878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.843885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.844169 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.844439 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.844442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.844444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.844613 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.845045 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.845051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.845293 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.845462 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.845758 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260812 11:20:48.848174 11969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.848688 11833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 3
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.189@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.190@KRBTEST.COM
I20260812 11:20:48.857283 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.849271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.30.255.188:45343)
0812 11:20:48.849369 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.849373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.849385 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:48.849414 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.849417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.849453 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.849529 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.849941 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.850678 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.851667 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.851961 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.852037 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.854074 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.854077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.854079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.854099 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.855568 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856116 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.856120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.856121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.856201 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856432 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.856435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.856437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.856623 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.856717 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.857058 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.857156 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:20:48.857290 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.849045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37534)
0812 11:20:48.849273 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.849286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.849354 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.849563 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.849566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.849742 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.849826 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.849833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.850794 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.850798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.851549 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.851570 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.852077 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.852081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.852163 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.852502 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.852524 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.853950 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.855678 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.855682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.855684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.856003 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.856270 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.856276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.856355 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.856713 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.856716 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.856812 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.857000 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.857142 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 11:20:48.857797 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.848681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.30.255.189:34633)
0812 11:20:48.848817 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.848822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.848855 (+ 33us) server_negotiation.cc:407] Connection header received
0812 11:20:48.848981 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.848984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.849039 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.849107 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.849588 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.850189 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.850917 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.851244 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.851334 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.854619 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.854624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.854626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.854652 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.856125 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856587 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.856590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.856592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.856668 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856923 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.856926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.856928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.857150 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.857239 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.857562 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.857668 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:20:48.858130 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.848575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.189:34633 (local address 127.0.0.1:43354)
0812 11:20:48.848801 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.848813 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.848892 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.849221 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.849224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.849377 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.849476 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.849481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.850291 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.850294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.850802 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.850811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.851472 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.851475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.851566 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.852109 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.852122 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.854434 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.856240 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856244 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.856245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.856483 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.856762 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.856767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.856837 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.857280 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.857283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.857370 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.857903 (+ 533us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.858011 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"spinlock_wait_cycles":3968}
I20260812 11:20:48.858807 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.848635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.30.255.190:41737)
0812 11:20:48.848728 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.848733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.849061 (+ 328us) server_negotiation.cc:407] Connection header received
0812 11:20:48.849156 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.849159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.849199 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.849265 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.849973 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.850695 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.851472 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.851701 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.851794 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.854644 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.854648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.854650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.854674 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.856558 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.856976 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.856980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.856982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.857070 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.857363 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.857367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.857368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.857583 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.857783 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.858570 (+ 787us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.858676 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:20:48.859071 11969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:20:48.860947 11833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: Bootstrap starting.
I20260812 11:20:48.862638 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } attempt: 3
I20260812 11:20:48.863830 11833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:48.864740 11833 log.cc:824] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.188@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.255.189@KRBTEST.COM
I20260812 11:20:48.866739 11833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: No bootstrap required, opened a new log
Aug 12 11:20:48 dist-test-slave-gjrz krb5kdc[11522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.255.190@KRBTEST.COM for kudu/127.30.255.188@KRBTEST.COM
I20260812 11:20:48.870260 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.859864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.30.255.189:34633)
0812 11:20:48.859982 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.859986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.860132 (+ 146us) server_negotiation.cc:407] Connection header received
0812 11:20:48.860236 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.860238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.860270 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.860336 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.860995 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.861552 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.862606 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.863445 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.863533 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.866157 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.866160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.866163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.866186 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.867791 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868436 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.868440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.868442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.868570 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868930 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.868934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.868935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.869134 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.869298 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.870017 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.870125 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:20:48.870416 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.859467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37548)
0812 11:20:48.860284 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.860297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.860364 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.860580 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.860583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.860730 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.860810 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.860817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.861774 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.861778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.862559 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.862570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.863134 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.863137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.863208 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.863805 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.863820 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.865911 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.868317 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.868322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.868549 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.868811 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868813 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.868815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.868887 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.869362 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.869365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.869698 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.869803 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.870277 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260812 11:20:48.870694 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.859574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.30.255.188:45343)
0812 11:20:48.859661 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.859665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.860424 (+ 759us) server_negotiation.cc:407] Connection header received
0812 11:20:48.860446 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.860448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.860477 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.860556 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.860988 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.861661 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.862645 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.862951 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.863040 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.866034 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.866038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.866040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.866060 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.868216 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868640 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.868642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.868644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.868723 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.868969 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.868972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.868974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.869190 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.869329 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.870168 (+ 839us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.870577 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":309,"server-negotiator.queue_time_us":28,"spinlock_wait_cycles":6272}
I20260812 11:20:48.870834 11833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.871060 11833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bce4f7a724e47be812b90ced7722cff, State: Initialized, Role: FOLLOWER
I20260812 11:20:48.871417 11833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.871906 11980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.872124 11980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.872740 11833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:48.872983 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.862983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.255.188:45343 (local address 127.0.0.1:37554)
0812 11:20:48.863135 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 11:20:48.863148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:20:48.863216 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:20:48.863692 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:20:48.863695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:20:48.863864 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:20:48.863966 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.863974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.865691 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.865694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.866450 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:20:48.866462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.867137 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.867141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:20:48.867233 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.867621 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:20:48.867637 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:20:48.869017 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:20:48.871528 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.871532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.871534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.871765 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.872074 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:20:48.872077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:20:48.872079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:20:48.872170 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:20:48.872532 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:20:48.872535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:20:48.872618 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 11:20:48.872702 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:20:48.872817 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 11:20:48.873008 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.863038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.30.255.188:45343)
0812 11:20:48.863455 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.863464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.863476 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:20:48.863509 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.863512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.863553 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.864143 (+ 590us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.864639 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.865507 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.866590 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.867008 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.867449 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.869144 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.869147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.869150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.869177 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.871410 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.871878 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.871881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.871883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.871969 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.872275 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.872278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.872280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.872429 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.872518 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.872767 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.872861 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"spinlock_wait_cycles":384,"thread_start_us":92,"threads_started":1}
I20260812 11:20:48.873839 11969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: Bootstrap starting.
I20260812 11:20:48.876274 11969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:48.877102 11969 log.cc:824] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:48.877617 31742 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 11:20:48.878362 12002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:48.878479 12002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:48.876273 11693 master_runner.cc:186] Error getting master registration for 127.30.255.190:41737: OK, instance_id { permanent_uuid: "0d9258cba5244f218489944454778546" instance_seqno: 1786533648401766 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:20:48.879365 11693 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:48.879722 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: Bootstrap starting.
I20260812 11:20:48.878758 11638 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0d9258cba5244f218489944454778546"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48924
I20260812 11:20:48.881726 11683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: Neither blocks nor log segments found. Creating new log.
I20260812 11:20:48.882411 11683 log.cc:824] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: Log is configured to *not* fsync() on all Append() calls
I20260812 11:20:48.884022 11969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: No bootstrap required, opened a new log
I20260812 11:20:48.884481 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: No bootstrap required, opened a new log
I20260812 11:20:48.887142 11683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.887380 11683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9258cba5244f218489944454778546, State: Initialized, Role: FOLLOWER
I20260812 11:20:48.887683 11683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.887710 11969 raft_consensus.cc:348] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.887914 11969 raft_consensus.cc:729] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac70f12fe704d86a69a512aca21bc5c, State: Initialized, Role: FOLLOWER
I20260812 11:20:48.888217 11969 consensus_queue.cc:260] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.888305 12011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.888629 12011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.888895 11683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:48.888973 11638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 11:20:48.889081 11638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:48.889318 11969 sys_catalog.cc:564] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:20:48.890206 11638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
I20260812 11:20:48.891754 11786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d9258cba5244f218489944454778546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8bce4f7a724e47be812b90ced7722cff"
I20260812 11:20:48.891979 11786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:48.891798 12012 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.891947 11928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d9258cba5244f218489944454778546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dac70f12fe704d86a69a512aca21bc5c"
I20260812 11:20:48.892985 11928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:20:48.893157 11786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9258cba5244f218489944454778546 in term 1.
I20260812 11:20:48.893764 11596 leader_election.cc:302] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9258cba5244f218489944454778546, 8bce4f7a724e47be812b90ced7722cff; no voters:
I20260812 11:20:48.891108 11638 leader_election.cc:288] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [CANDIDATE]: Term 1 election: Requested vote from peers 8bce4f7a724e47be812b90ced7722cff (127.30.255.189:34633), dac70f12fe704d86a69a512aca21bc5c (127.30.255.188:45343)
I20260812 11:20:48.894135 11928 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9258cba5244f218489944454778546 in term 1.
I20260812 11:20:48.894332 12012 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 11:20:48.895247 12040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:48.895349 12040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 11:20:48.896883 12050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:20:48.896955 12050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:20:48.897821 12011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:20:48.898075 12011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9258cba5244f218489944454778546, State: Running, Role: LEADER
I20260812 11:20:48.898409 12011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } }
W20260812 11:20:48.898502 11600 tablet.cc:2369] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:20:48.900305 12011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d9258cba5244f218489944454778546. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.900899 11845 master_runner.cc:263] No leader master found from master dac70f12fe704d86a69a512aca21bc5c
I20260812 11:20:48.900964 11845 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:48.900921 12011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:48.901165 11546 master_runner.cc:263] No leader master found from master 0d9258cba5244f218489944454778546
I20260812 11:20:48.901226 11546 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:20:48.902951 12069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:20:48.903829 12069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:20:48.910326 11928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [term 1 FOLLOWER]: Refusing update from remote peer 0d9258cba5244f218489944454778546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:48.911293 12011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:48.916132 12012 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d9258cba5244f218489944454778546. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.916239 12012 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.916514 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.902361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.30.255.188:45343)
0812 11:20:48.902464 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.902468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.906419 (+ 3951us) server_negotiation.cc:407] Connection header received
0812 11:20:48.906945 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.906948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.906989 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.907035 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.907802 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.908594 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.909720 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.910096 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.910168 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.911962 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.911978 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.911981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.912005 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.913828 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.915010 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.915013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.915015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.915099 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.915421 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.915425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.915427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.915630 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.915761 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.916094 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.916203 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 11:20:48.918916 11786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [term 1 FOLLOWER]: Refusing update from remote peer 0d9258cba5244f218489944454778546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:20:48.919517 12011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:20:48.920327 12061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.920442 12061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:48.921444 12012 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.921553 12012 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.921962 12081 mvcc.cc:205] Tried to move back new op lower bound from 7317641825929867264 to 7317641825892384768. Current Snapshot: MvccSnapshot[applied={T|T < 7317641825929867264}]
I20260812 11:20:48.923537 11980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d9258cba5244f218489944454778546. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.923652 11980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.923746 11980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.923816 11980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:20:48.923794 12082 mvcc.cc:205] Tried to move back new op lower bound from 7317641825929867264 to 7317641825892384768. Current Snapshot: MvccSnapshot[applied={T|T < 7317641825929867264}]
I20260812 11:20:48.925887 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.900791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.30.255.189:34633)
0812 11:20:48.900888 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.900892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.901729 (+ 837us) server_negotiation.cc:407] Connection header received
0812 11:20:48.901931 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.901935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.901996 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.902055 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.903040 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.903817 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.904993 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.905321 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.905395 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.912332 (+ 6937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.912336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.912338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.912363 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.914554 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.916912 (+ 2358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.916915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.916917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.917013 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.924990 (+ 7977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.924994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.924995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.925208 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.925299 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.925635 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.925748 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 11:20:48.928795 12090 mvcc.cc:205] Tried to move back new op lower bound from 7317641825929867264 to 7317641825892384768. Current Snapshot: MvccSnapshot[applied={T|T < 7317641825929867264}]
I20260812 11:20:48.928836 12069 catalog_manager.cc:1347] Generated new cluster ID: 614cb08d1dd5485382d402467620e77c
I20260812 11:20:48.928885 12069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:20:48.930547 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:48.901810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.30.255.190:41737)
0812 11:20:48.901918 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:48.901922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:48.906097 (+ 4175us) server_negotiation.cc:407] Connection header received
0812 11:20:48.906502 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:48.906505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:48.906550 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:48.906596 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:48.917306 (+ 10710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.918047 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.924956 (+ 6909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:48.925296 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:48.925396 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:48.927219 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:20:48.927223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:20:48.927225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:20:48.927251 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:20:48.928779 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.929333 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.929337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.929338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.929425 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:20:48.929742 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:20:48.929745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:20:48.929747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:20:48.929922 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:20:48.930013 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:48.930307 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 11:20:48.930405 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:20:48.973783 12087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d9258cba5244f218489944454778546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9258cba5244f218489944454778546" member_type: VOTER last_known_addr { host: "127.30.255.190" port: 41737 } } peers { permanent_uuid: "8bce4f7a724e47be812b90ced7722cff" member_type: VOTER last_known_addr { host: "127.30.255.189" port: 34633 } } peers { permanent_uuid: "dac70f12fe704d86a69a512aca21bc5c" member_type: VOTER last_known_addr { host: "127.30.255.188" port: 45343 } } }
I20260812 11:20:48.973945 12087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546 [sys.catalog]: This master's current role is: LEADER
I20260812 11:20:48.981065 12069 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:20:48.982295 12069 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:20:48.999286 12069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d9258cba5244f218489944454778546: Generated new TSK 0
I20260812 11:20:49.000017 12069 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:20:49.026470 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.023587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.30.255.189:34633)
0812 11:20:49.023707 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.023711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.025438 (+ 1727us) server_negotiation.cc:407] Connection header received
0812 11:20:49.025622 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.025626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.025674 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.025726 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.026340 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43384: BlockingRecv error: recv got EOF from 127.0.0.1:43384 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 11:20:49.027850 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.024372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.30.255.190:41737)
0812 11:20:49.024480 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.024485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.026941 (+ 2456us) server_negotiation.cc:407] Connection header received
0812 11:20:49.027108 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.027111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.027160 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.027214 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.027739 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48934: BlockingRecv error: recv got EOF from 127.0.0.1:48934 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 11:20:49.029202 11978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.024969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.30.255.188:45343)
0812 11:20:49.025074 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.025078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.028228 (+ 3150us) server_negotiation.cc:407] Connection header received
0812 11:20:49.028526 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.028529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.028579 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.028627 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.029096 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37562: BlockingRecv error: recv got EOF from 127.0.0.1:37562 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 11:20:49.033172 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.031938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.30.255.189:34633)
0812 11:20:49.032043 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.032047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.032281 (+ 234us) server_negotiation.cc:407] Connection header received
0812 11:20:49.032480 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.032483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.032552 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.032601 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.033081 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43388: BlockingRecv error: recv got EOF from 127.0.0.1:43388 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 11:20:49.037897 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.036571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.30.255.189:34633)
0812 11:20:49.036663 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.036667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.037106 (+ 439us) server_negotiation.cc:407] Connection header received
0812 11:20:49.037274 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.037277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.037324 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.037369 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.037812 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43392: BlockingRecv error: recv got EOF from 127.0.0.1:43392 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 11:20:49.043169 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.041783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.30.255.189:34633)
0812 11:20:49.041897 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.041900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.042382 (+ 482us) server_negotiation.cc:407] Connection header received
0812 11:20:49.042548 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.042551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.042596 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.042641 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.043087 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43394: BlockingRecv error: recv got EOF from 127.0.0.1:43394 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 11:20:49.050359 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.048967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.30.255.189:34633)
0812 11:20:49.049088 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.049091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.049555 (+ 464us) server_negotiation.cc:407] Connection header received
0812 11:20:49.049727 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.049730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.049775 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.049818 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.050276 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43400: BlockingRecv error: recv got EOF from 127.0.0.1:43400 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 11:20:49.061703 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.060218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.30.255.189:34633)
0812 11:20:49.060314 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.060318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.060817 (+ 499us) server_negotiation.cc:407] Connection header received
0812 11:20:49.061009 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.061012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.061061 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.061107 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.061615 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43410: BlockingRecv error: recv got EOF from 127.0.0.1:43410 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 11:20:49.081727 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.079866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.30.255.189:34633)
0812 11:20:49.079983 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.079987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.080614 (+ 627us) server_negotiation.cc:407] Connection header received
0812 11:20:49.080932 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.080936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.080992 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.081045 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.081629 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43416: BlockingRecv error: recv got EOF from 127.0.0.1:43416 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 11:20:49.117072 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.116093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.30.255.189:34633)
0812 11:20:49.116218 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.116221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.116353 (+ 132us) server_negotiation.cc:407] Connection header received
0812 11:20:49.116469 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.116471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.116519 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.116600 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.116966 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43428: BlockingRecv error: recv got EOF from 127.0.0.1:43428 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 11:20:49.184226 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.183057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.30.255.189:34633)
0812 11:20:49.183200 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.183205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.183380 (+ 175us) server_negotiation.cc:407] Connection header received
0812 11:20:49.183545 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.183548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.183598 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.183666 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.184110 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43430: BlockingRecv error: recv got EOF from 127.0.0.1:43430 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 11:20:49.315294 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.314225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.30.255.189:34633)
0812 11:20:49.314412 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.314416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.314504 (+ 88us) server_negotiation.cc:407] Connection header received
0812 11:20:49.314632 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.314634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.314684 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.314756 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.315145 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43432: BlockingRecv error: recv got EOF from 127.0.0.1:43432 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260812 11:20:49.574482 11844 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:20:49.573368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.30.255.189:34633)
0812 11:20:49.573517 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:49.573522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:49.573629 (+ 107us) server_negotiation.cc:407] Connection header received
0812 11:20:49.573775 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:49.573778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:49.573831 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:49.573902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:20:49.574362 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43434: BlockingRecv error: recv got EOF from 127.0.0.1:43434 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:20:49.879907 12002 catalog_manager.cc:1259] Loaded cluster ID: 614cb08d1dd5485382d402467620e77c
I20260812 11:20:49.879993 12002 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: loading cluster ID for follower catalog manager: success
I20260812 11:20:49.881738 12002 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: acquiring CA information for follower catalog manager: success
I20260812 11:20:49.882956 12002 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8bce4f7a724e47be812b90ced7722cff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:20:49.897920 12050 catalog_manager.cc:1259] Loaded cluster ID: 614cb08d1dd5485382d402467620e77c
I20260812 11:20:49.897979 12050 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: loading cluster ID for follower catalog manager: success
I20260812 11:20:49.899521 12050 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: acquiring CA information for follower catalog manager: success
I20260812 11:20:49.900595 12050 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dac70f12fe704d86a69a512aca21bc5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:20:50.093111 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:50.088847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.30.255.189:34633)
0812 11:20:50.089104 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:50.089109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:50.089248 (+ 139us) server_negotiation.cc:407] Connection header received
0812 11:20:50.089418 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:50.089421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:50.089626 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:50.089730 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:50.090077 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.090613 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.091610 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.091858 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.091947 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:50.092278 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:50.092714 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:50.092824 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:50.092897 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 11:20:50.092954 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260812 11:20:50.099545 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:50.096069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.30.255.188:45343)
0812 11:20:50.096298 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:50.096303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:50.096317 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:20:50.096441 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:50.096444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:50.096553 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:50.096651 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:50.096865 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.097382 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.098182 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.098412 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.098497 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:50.098779 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:50.099146 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:50.099250 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:50.099355 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 11:20:50.099411 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":64,"threads_started":1}
I20260812 11:20:50.105818 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:50.102195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.30.255.189:34633)
0812 11:20:50.102298 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:50.102302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:50.102738 (+ 436us) server_negotiation.cc:407] Connection header received
0812 11:20:50.102910 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:50.102913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:50.102955 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:50.103027 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:50.103366 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.104085 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.104853 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.105091 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.105183 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:50.105359 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:50.105432 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:50.105494 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:50.105648 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 11:20:50.105697 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:20:50.106529 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:50.102169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.30.255.190:41737)
0812 11:20:50.102690 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:50.102694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:50.102709 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:20:50.102791 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:50.102794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:50.102924 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:50.103042 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:50.103213 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.103789 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.105077 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.105374 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.105476 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:50.105841 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:50.106169 (+ 328us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:50.106261 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:50.106325 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 11:20:50.106388 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":62,"threads_started":1}
I20260812 11:20:50.106773 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:20:50.102500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.30.255.188:45343)
0812 11:20:50.102595 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 11:20:50.102599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:20:50.102634 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:20:50.102754 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:20:50.102757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:20:50.102801 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:20:50.102870 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:20:50.103161 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.103908 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.105553 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:20:50.105831 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:20:50.105919 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:20:50.106241 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:20:50.106313 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:20:50.106370 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:20:50.106639 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 11:20:50.106687 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 11:20:50.109210 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11546
I20260812 11:20:50.125710 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11693
I20260812 11:20:50.133140 31742 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoRXcNR/build/debug/bin/kudu with pid 11845
2026-08-12T11:20:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3227 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6435 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284560 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST